How to: Write Automated Tests for Drupal

With an automated testing framework in core, Drupal is now far along the road to a practice of Test-driven development. But there's one thing missing: a complete, in-depth, up-to-date tutorial on how to actually write these tests. Although there is the SimpleTest Automator module to help you create these tests through the user interface, it is currently imperfect and under development, so this post will be a tutorial on writing these tests manually.

Testing resources

Here's just a general list of resources we should keep handy while writing our test:

  • Testing API functions - This is a quick cheat sheet of some functions that are very helpful during testing. These include controlling the internal testing browser, creating users with specific permissions, and simulating clicking on links that appear in the page.
  • Available assertions - Assertions are how you determine whether your code is working or not - you assert that it should be working, and let the testing framework handle the rest for you. This is a library on the assertions that are available to use in our testing framework.

Know what you're testing

In this example, I will be testing Drupal's ability to change the site-wide theme. I know how to test this manually - I would go to the admin/build/themes page, and select the radio button of a different default theme, and then make sure the theme had changed when I reload the page. In order to automate this test, I will simply write code to repeat the same actions I would have done manually.

Start writing your test case

Ok, now we get to the code. First of all, every test case will be a class that extends the DrupalWebTestCase class. So we'll start out with this code:

<?php
// $Id$

class ThemeChangingTestCase extends DrupalWebTestCase {

}
?>

Now, we have to tell the testing framework a little bit about our test. We give it three pieces of information - the name of the test, a brief description of the test, and the group of tests this test is a part of. We will return this information in our implementation of getInfo() in our test class:

<?php
// $Id$

class ThemeChangingTestCase extends DrupalWebTestCase {

  /**
   * Implementation of getInfo().
   */
 
function getInfo() {
    return array(
     
'name' => t('Changing the site theme.'),
     
'description' => t('Tests the changing of the site-wide theme through the administration pages.'),
     
'group' => t('Theming'),
    );
  }

}
?>

Now, let's add our test function. The most thorough way to test this would be to cycle through all the themes, setting each in turn as the default, and that's what we'll do. So we'll get this:

<?php
// $Id$

class ThemeChangingTestCase extends DrupalWebTestCase {
 
// We need a list of themes to cycle through.
 
var $themes = array('bluemarine', 'chameleon', 'garland', 'marvin', 'minnelli', 'pushbutton');

  /**
   * Implementation of getInfo().
   */
 
function getInfo() {
    return array(
     
'name' => t('Changing the site theme.'),
     
'description' => t('Tests the changing of both the site-wide theme and the user-specific theme, and makes sure that each continues to work properly.'),
     
'group' => t('Theming'),
    );
  }

  /**
   * This is the function where we will actually do the testing.
   * It will be automatically called, so long as it starts with the lowercase 'test'.
   */
 
function testSitewideThemeChanging() {
    foreach (
$this->themes as $theme) {
     
// We need a test user with permission to change the site-wide theme.
     
$user = $this->drupalCreateUser(array('administer site configuration'));
     
$this->drupalLogin($user);
     
// Now, make a POST request (submit the form) on the admin/build/themes page.
     
$edit = array();
      foreach (
$this->themes as $theme_option) {
       
$edit["status[$theme_option]"] = FALSE;
      }
     
$edit["status[$theme]"] = TRUE;
     
$edit['theme_default'] = $theme;
     
$this->drupalPost('admin/build/themes', $edit, t('Save configuration'));
    }
  }
}
?>

Whoa, that's a lot of code. Let's go through what I've done step by step:

  1. I've declared a class variable that lists the themes that we're going to test.
  2. In my test function, I'm cycling through each theme, in turn setting each one as the default.
  3. I create a user with enough permissions to change the site-wide theme.
  4. After logging this user in, I proceed to make a POST request to the admin/build/themes page, enabling only the theme we're testing, and setting that theme to be the theme default.
  5. I then submit the form by clicking on the 'Save configuration' button.

Now this works very well, but one thing is still missing - how am I sure that the theme has changed? If I were doing this manually I could tell by just looking - but when I'm automating this, I will test for the theme's css files in the page source of the reloaded admin/build/themes page upon submission. To do this, I will use the assertRaw() function, which makes sure that some text is found in the raw HTML of the current page in the internal browser:

<?php
// $Id$

class ThemeChangingTestCase extends DrupalWebTestCase {
 
// We need a list of themes to cycle through.
 
var $themes = array('bluemarine', 'chameleon', 'garland', 'marvin', 'minnelli', 'pushbutton');

  /**
   * Implementation of getInfo().
   */
 
function getInfo() {
    return array(
     
'name' => t('Changing the site theme.'),
     
'description' => t('Tests the changing of both the site-wide theme and the user-specific theme, and makes sure that each continues to work properly.'),
     
'group' => t('Theming'),
    );
  }

  /**
   * This is the function where we will actually do the testing.
   * It will be automatically called, so long as it starts with the lowercase 'test'.
   */
 
function testSitewideThemeChanging() {
    foreach (
$this->themes as $theme) {
     
// We need a test user with permission to change the site-wide theme.
     
$user = $this->drupalCreateUser(array('administer site configuration'));
     
$this->drupalLogin($user);
     
// Now, make a POST request (submit the form) on the admin/build/themes page.
     
$edit = array();
      foreach (
$this->themes as $theme_option) {
       
$edit["status[$theme_option]"] = FALSE;
      }
     
$edit["status[$theme]"] = TRUE;
     
$edit['theme_default'] = $theme;
     
$this->drupalPost('admin/build/themes', $edit, t('Save configuration'));
     
// Make sure we've actually changed themes.
     
$this->assertCSS($theme);
    }
  }

  /**
   * Custom assert method - make sure we actually have the right theme enabled.
   *
   * @param $theme
   *   The theme to check for the css of.
   * @return
   *   None.
   */
 
function assertCSS($theme) {
   
// Minnelli is the only core theme without a style.css file, so we'll use
    // minnelli.css as an indicator instead.
   
$file = $theme == 'minnelli' ? 'minnelli.css' : 'style.css';
   
$this->assertRaw(drupal_get_path('theme', $theme) . "/$file", t("Make sure the @theme theme's css files are properly added.", array('@theme' => $theme)));
  }
}
?>

Note that I've added my own assertCSS() function in the above code. You're perfectly free to add whatever functions you may desire—just keep in mind that if they start with the lowercase 'test', they will be automatically called as a test function!

That concludes the basic tutorial. Read on if you're interested in going beyond the basics! :)

Advanced testing techniques

Here I'll go over a few techniques for better, easier, simpler, and just overall awesomer testing.

  • Using setUp() and/or tearDown() methods.

    Sometimes it can be useful to run some code before and/or after our test methods are finished running. To do this, we can implement setUp() and/or tearDown(), which run before and after the test methods, respectively:

    <?php
    // $Id$

    class ThemeChangingTestCase extends DrupalWebTestCase {

      /**
       * Implementation of setUp().
       */
     
    function setUp() {
       
    // Invoke setUp() in our parent class, to set up the testing environment.
       
    parent::setUp();
       
    // Create and log in our test user.
       
    $user = $this->drupalCreateUser(array('administer site configuration'));
       
    $this->drupalLogin($user);
      }
    }
    ?>

    In the above example, we've created and logged in our test user in our setUp() method. As a result, that user will be logged in for our test methods; this can be considered somewhat cleaner code.

    We can also use setUp() to enable additional modules we may need:

    <?php
    // $Id$

    class ThemeChangingTestCase extends DrupalWebTestCase {

      /**
       * Implementation of setUp().
       */
     
    function setUp() {
       
    // Invoke setUp() in our parent class, to set up the testing environment.
        // We're going to test the theming of the search box and the tracker page,
        // so we need those modules enabled.
       
    parent::setUp('search', 'tracker');
       
    // Create and log in our test user.
       
    $user = $this->drupalCreateUser(array('administer site configuration'));
       
    $this->drupalLogin($user);
      }
    }
    ?>

    Note: Make sure to always call parent::setUp() and parent::tearDown() if you override them! If you don't, the testing framework will either fail to be set up, or fail to be teared down, successfully.

  • Dealing with HTML content using SimpleXML.

    Our assertCSS() function in the basic example is far from ideal. The path to the theme's style.css could appear on the page as plain text (not part of a css link), and the test would still pass.

    To get around this weakness, we can handle the HTML content of the fetched page using SimpleXML. For example:

    <?php
    // $Id$

    class ThemeChangingTestCase extends DrupalWebTestCase {

      /**
       * Custom assert method - make sure we actually have the right theme enabled.
       *
       * @param $theme
       *   The theme to check for the css of.
       * @return
       *   None.
       */
     
    function assertCSS($theme) {
       
    // Minnelli is the only core theme without a style.css file, so we use
        // minnelli.css as an indicator instead.
       
    $file = $theme == 'minnelli' ? 'minnelli.css' : 'style.css';
        if (
    $this->parse()) {
         
    $links = $this->elements->xpath('//link');
          foreach (
    $links as $link) {
            if (
    strpos($link['href'], base_path() . drupal_get_path('theme', $theme) . "/$file") === 0) {
             
    $this->pass(t("Make sure the @theme theme's css files are properly added.", array('@theme' => $theme)));
              return;
            }
          }
         
    $this->fail(t("Make sure the @theme theme's css files are properly added.", array('@theme' => $theme)));
        }
      }
    }
    ?>

    The parse() method must be called in order to populate $this->elements. It is not called automatically on every page load because this would lead to a significant performance drain.

  • Creating an unused base class, and then extending it.

    Sometimes, it would make sense for two test cases to share API functions, or even setUp and tearDown() functions. In order to do this, we'll set up one base test case that extends DrupalWebTestCase, and then create several children test cases that extend our base test case. For example:

    <?php
    // $Id$

    class ThemeChangingTestCase extends DrupalWebTestCase {

      function assertCSS($theme) {
       
    // ...
     
    }
    }

    class SiteWideThemeChangingTestCase extends ThemeChangingTestCase {

      /**
       * Implementation of getInfo().
       */
     
    function getInfo() {
       
    // ...
     
    }
    }

    class UserSpecificThemeChangingTestCase extends ThemeChangingTestCase {

      /**
       * Implementation of getInfo().
       */
     
    function getInfo() {
       
    // ...
     
    }
    }
    ?>

    Note that in order for a test case to not actually be run itself (or even show up on the administration interface), all it has to do is not implement getInfo().

Now you are officially qualified to start writing tests! If you're looking for a place to get started, check out the issue queue. Also, the code coverage reports are a great place to see what needs to be tested in core.

Comments

Testing in Ubercart

Hey Charlie, thanks for the write-up. Myself and the other Uberdevs intend to get testing into Ubercart as soon as possible. To that end, I'm curious if this is even possible in Drupal 6. I understand there was a SimpleTest module for D6, but if we start writing tests based on that for our D6 work will it be a pain to then convert those to Drupal's native solution once we move onto D7?

Thanks for all the work you're putting into testing. : )

As of now

As of now, it's the same API for the Drupal 7 core module and the Drupal 6 contributed module (http://drupal.org/project/simpletest), so this tutorial should work for both.

D6 vs. D7

I've noticed there are some API changes such as D7 extends DrupalWebTestCase while d6 extends DrupalTestCase. hook_getInfo() vs. hook_get_info(), etc.

No...

You are mistaken. Since perhaps 3 months ago, Drupal 6 SimpleTest supports the new DrupalWebTestCase API, as well as hook_getInfo().

PHP5 standards

Since Drupal7 requires PHP5, we can use public, protected and private to declare class properties instead of var. For test cases, we can generally use protected so that properties are only available to the test itself and any test who extend it:

<?php
class ThemeChangingTestCase extends DrupalWebTestCase {
// We need a list of themes to cycle through.
protected $themes = array('bluemarine', 'chameleon', 'garland', 'marvin', 'minnelli', 'pushbutton');
...
?>

Thanks, a couple of queries

Thanks for the helpful writeup.

1. I think I've convinced myself (re. the start of the Advanced testing section) that DrupalWebTestCase implements its own setUp() and tearDown() itself and these are automatically called when any test is run - if I got it right perhaps this could be added in the main text above?

2. In your SimpleXML example you use $this->pass and $this->fail. Where are these documented (and presumably many other useful methods)? The links at the top of the page to the lists of Testing API functions and Available assertions only seem to give a few.

Ta,
gpk

lower the barriers!

Patch reviewing environment is a fantastic idea. Making it easy to test patches will do wonders for getting the many old languishing patches out there dealt with.

Out of curiosity what is the current code coverage of core tests?

Thanks for all your work on this. This whole thing is really helping to bring the quality of Drupal head and shoulders above the competition.

Post new comment

The content of this field is kept private and will not be shown publicly.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Allowed HTML tags: <a> <em> <strong> <cite> <code> <ul> <ol> <li> <dl> <dt> <dd>
  • Lines and paragraphs break automatically.

More information about formatting options

You must answer the above question correctly to be able to submit a comment. This measure is designed to protect this site against spambots, which seem able to circumvent even the toughest image captchas. Hint: try googling for the question answer if you really don't know it. The theory behind this is that if this website is running its own completely custom captcha, it will not be in anyone's economic advantage to rewrite their spam bot's code to handle this website's questions. Powered by CwgordonCaptcha.