This is part of an ongoing series of posts about how to get started writing tests for Ruby on Rails. The series begins with the introduction and overview of the ideas behind testing.

Unit Testing

I am going to cover several types of tests over the course of this tutorial: unit tests, functional tests, integration tests. I will start with unit tests. Unit tests are a good way to introduce the subject of testing and, if you already have a working Rails application, they are the first tests you should consider adding. Think of unit tests as our first testing layer, a foundation for the other layers.

Unit testing is a testing technique to validate that individual “units” of code are working properly. A “unit” refers to the smallest testable part of the code. When unit testing, we are not interested in how all the units fit together, just in make sure that each unit is functioning properly on its own.

It might be helpful to use a real world metaphor to illustrate this point. If we were unit testing a car battery, we would test the battery in near isolation to see if it behaves as a battery ought to behave. We could test that the battery voltage was 12-12.6 V, that it has enough electrolyte solution, that the electrolyte solution has the correct ratio of sulfuric acid and water, and that the battery is grounded. We would also want to test that the battery recharges properly, which might require us to connect it to the alternator. In that case, we wouldn’t be interested in examining the alternator’s behavior, only the battery’s. We would assume that the alternator would get a thorough unit testing of its own. In fact, it would be even better if we could find a way to substitute in reliable test equipment in place of the alternator to simulate its effect on the battery. Then we could remove the alternator from the battery’s unit tests altogether.

What are the “units” being tested in this example? They are not the car’s electrical system, the battery, or even the small parts that make up the battery. They are each of the characteristics or functions of the battery. Testing for the correct voltage is one unit test. Testing the electrolyte ratio is another unit test. Testing that it recharges correctly is another unit test. But taken together we can say that we are “unit testing the battery”. The battery is not the unit, but it is the object of all of our unit tests.

Choosing good unit tests to run requires thinking about what the object’s characteristics and functions should be. What properties should a car battery have? What should it be able to do if working properly? What would be the indications that something was wrong with the battery? You might wonder if we should also test even smaller parts, like the sulfuric acid in the electrolyte solution, or test for more fundamental assumptions, such whether the battery is in the shape of a box. You could test these things, without a doubt. But I would say that sulfuric acid is fundamental enough that it is safe to assume it works like all sulfuric acid, and the shape of the battery can be safely presumed or seen as unimportant. We could drive ourselves crazy trying to test everything, and then we would lose sight of the main point—to make sure that the battery we have (whatever its shape, color, contents) does the job we need. We don’t really care if the battery is shaped like an egg, as long as it puts out the voltage we need, recharges properly, and so on.

Once we apply unit testing to Ruby objects, choosing good unit tests is easy. The object of our tests will be a Class (or an instance of a Class). The Class’s characteristics and functions will be spelled out for us as attributes and methods. In most cases, each method will get at least one unit test.

Ruby TestUnit in IRB

To begin our hands-on adventures in testing, we will start by learning how easy it is to use TestUnit with basic Ruby code. After that, we will be able to apply those same techniques to our Rails application. TestUnit (or Test::Unit) is a unit testing framework that is part of the standard Ruby distribution. If you have Ruby installed, you already have TestUnit available for use.

There are three basic steps to using TestUnit. 1) Include the TestUnit library so that you can use it (require ‘test/unit’). 2) Define a new Class that inherits from Test::Unit::TestCase so that your tests will inherit all of TestUnit’s “testing behaviors” (the same way your Models inherit ActiveRecord’s behaviors). 3) Inside your new Class, define methods where each method will be a unit test containing one or more assertions. The method name of each test will begin with “test_”.

As for writing assertions in your methods, let’s start with the simplest one: assert(boolean_statement). If the boolean_statement is true the test passes. If it is false, it fails.

Let’s see how it works from a command line using IRB (Interactive Ruby):

We followed our three steps for writing a test class and giving it methods and assertions. Notice that I intentionally wrote one assertion that would pass and one that would fail. When we finished the definition, nothing happened. When we exited IRB, then our tests ran. That’s a characteristic of using IRB for our tests and won’t be significant as we move on.

“Started” marks the beginning of our tests and the line below that says “.F”. The period indicates a test that ran and passed. The “F” indicates a test that ran and failed. So “..FF.” would have indicated two passed tests, followed by two failed tests, followed by a passed test. “E” would indicate an error instead of a failed assertion. Of course, we’ll be trying to write tests that report all periods.

After it reports how long the tests took, it outputs details on any failures. In our case, we had a failure on line 8: it was expecting “true” but the result of the assertion was “false”. (The “<>” are just there to highlight the actual result and don’t have another meaning.) This information will help us to locate which of our tests are failing and become an indispensable tool when debugging.

The tests conclude with a summary showing how many tests, assertions, failures and errors. Our goal will be to have tests that all pass and give us “0 failures, 0 errors” on that line. (Don’t be surprised when you catch yourself chanting “Come on double-zero!” while your tests are running. We all do it!)

Finally, IRB does an ungraceful exit. It is a quirk of IRB and one that we can bypass from now on by putting our tests into a text file.

Ruby TestUnit on a File

Create a text file and save it as “first_tests.rb”. Into that file, put in the same code that we typed into IRB above (including the “require” statement), but this time, reverse the order of the two methods. Now run your tests from the command line:

This time the tests run immediately.

Not only did we avoid IRB’s ungraceful exit, and save ourselves some typing in the future, but we can also observe another point about how tests work. Notice that the results still show “.F”. Even though we reversed the order of the test methods, it did not return “F.” Tests are executed in alphabetical order, not in the order that they are defined. So “test_addition” is being run first, then “test_subtraction”. A method called “test_multiplication” would run between the two, regardless of where you put the code to define it.

More Unit Tests, setup and assert_equal

Let’s look at some more complex unit tests assertions and add another assertion type to our toolbox.

Here is a revised version of first_test.rb.

Before you run the tests, examine the changes we made.

1. We added a special method called “setup”. The setup method runs before each and every test. (Similar to “initialize”.) Our setup method simply sets two instance variables which, because they are instance variables (with the “@” in front), are available for use in each of our test methods.

2. test_subtraction now contains two assertions. The first should fail, the second should succeed.

3. The assert() in test_addition now includes a second optional argument, a message to display if the test fails.

4. Two new tests were added: test_array_a and test_array_b. Both make use of @array from the setup method. Notice that test_array_a tests the length of the array, then makes a change (adds a value), then tests that the result is what was expected. This is a common pattern in testing.

5. Test_array_a and test_array_b also use a new assertion, assert_equal(expected, actual). You will use assert_equal more often than assert. It is no different than writing assert(expected == actual). The advantage of this format is that when the test fails it displays the expected and actual values for your review instead of just saying “<false> is not true”. Because of the way the default failure message renders, you will want to always put the expected value first. As you can see in test_array_b it also allows for an optional failure message. (All assertions accept an optional message argument.)

6. Notice that reverse_array is just a method which is called by both test_array_a and test_array_b. It’s no problem to make use of “helper methods” like this. Because the method name does not start with “test_” it will not be run as a test.

7. test_array_b is included to illustrate that the changes made to @array in test_array_a are isolated and do not have an effect on other methods. Each test is run independently and calls the setup method before it begins.

8. test_string makes two assertions. The first assertion should return true, while the second will attempt to apply the length method to nil, resulting in an error. I want you to see how an error is different from a failure. A failure is a failed assertion, while an error means something went wrong when trying to execute your test before it could get to an assertion.

9. Finally, notice that the method “dont_test_string_content” should not be run as a test because it does not begin with “test_”. I use this technique when I want to temporarily disable a test.

Now run the tests again with ruby first_test.rb. It should report “5 tests, 7 assertions, 1 failures, 1 errors”.

But wait… go back and count how many assertions we were expecting to run. There were eight. This raises a good point: Tests cease execution when they encounter a failure. In test_subtraction, only the first assertion is tested. That assertion fails and therefore it never tests the second assertion. A failure in a test does not mean there is only one problem in our code that needs to be resolved; there could be more, but each test only reports the first one.

Play around with these tests on your own until you feel like you have an understanding of how they work.

If you are using TextMate you should note that the short cut for assert is “as<tab-key>” and the short cut for assert_equal is “ase<tab-key>”. (Extremely handy.)

UPDATE: The series continues in Part 2.

This entry was posted on Wednesday, November 14th, 2007 at 12:56 pm and is filed under Ruby on Rails, Tutorials. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.