Crossing borders: Testing in integrated frameworks, Part 1

Document options Document options requiring JavaScript are not displayed Learn and share! Exchange know-how with your peers -- try our new Pass It Along beta app Rate this page Help us improve this content

Level: Intermediate

Bruce Tate (bruce.tate@j2life.com), President, RapidRed

06 Jun 2006

The Java™ community has done a fantastic job of advancing automated unit testing. An increasing number of open source frameworks let you build automated test suites along with your projects. The Spring framework, JUnit, TestNG, and several other frameworks owe some or all of their inspiration to the idea of automated testing. Still, some non-Java languages and frameworks have more motivation to test, more suitable testing tools, and a more unified view of testing. By looking at how other frameworks test, you can improve the way you test in the Java language, or even use a more appropriate language to test your Java code. This article, the first of two on testing in Ruby on Rails, covers the Rails approach to unit testing.

Catching the bug

I remember when I first got the automated-testing bug. Mike Clark -- automated-testing guru in the Java community, author of the JUnitPerf performance-tuning tool (see Resources), and now a Ruby on Rails expert -- approached me at a conference after my presentation called Bitter Java. Mike told me about a way I could improve the presentation by automated testing. I followed him around for the rest of the conference and watched as many of his testing sessions as I could. I started using the techniques he advocated and got addicted to the idea of turning red bars (for failing tests) into green (for passing tests). Automated testing changed the way I think about developing software.

About this series In the Crossing borders series, author Bruce Tate advances the notion that today's Java programmers are well served by learning other approaches and languages. The programming landscape has changed since Java technology was the obvious best choice for all development projects. Other frameworks are shaping the way Java frameworks are built, and the concepts you learn from other languages can inform your Java programming. The Python (or Ruby, or Smalltalk, or ... fill in the blank) code you write can change the way that you approach Java coding. This series introduces you to programming concepts and techniques that are radically different from, but also directly applicable to, Java development. In some cases, you'll need to integrate the technology to take advantage of it. In others, you'll be able to apply the concepts directly. The individual tool isn't as important as the idea that other languages and frameworks can influence developers, frameworks, and even fundamental approaches in the Java community.

The Java community definitely has the automated-testing bug. Frankly, we don't have any choice. Competitive pressures force many organizations to write more code with fewer testers, yet each developer must still be more productive. If you don't automate, you test less -- not a viable alternative given the increasing complexity of modern applications.

We've seen an explosion of testing tools and techniques over the past 10 years. JUnit and TestNG are both good tools that enable automated unit testing, driven by the everyday developer. Selenium is a tool that improves integration and functional testing. A new family of development processes known as agile techniques teach you to emphasize automated testing more and to lean less on formal design artifacts as the sole tool for improving quality. The Java community has come a long way. (See Resources for additional information on tools and techniques discussed here.)

Other programming communities have the bug too. Some of them use automated testing even more than Java developers, with quite different dimensions to their automated-testing experiences:

• Smalltalk programmers have used automated testing for almost 30 years, so some of the techniques used by dynamically typed languages are more advanced.
  
  
• Integrated framework developers have the advantage of knowing about the structure and composition of a framework's elements. Some frameworks, such as Ruby on Rails, can generate test cases and provide testing features by default.
  
  
• Languages with advanced metaprogramming capabilities, such as Ruby and Lisp, allow some testing tricks that other languages don't, such as easier access to mock objects.

In this article and the next, you'll get a complete understanding of how testing works within the Ruby on Rails integrated development framework. Part 1 focuses on testing model objects and gives you some Rails-inspired strategies you can use to make your Java unit testing more productive. Part 2 spends more time on functional tests and integration tests. As a Java programmer, some of the ideas will be familiar to you, particularly if you test, and others will stretch your understanding.

Back to top

Plugging a hole

In the previous installment in this series, you learned that dynamic typing enables certain kinds of bugs that a statically typed language would catch at compile time. The Ruby code fragment in Listing 1 contains four different bugs that won't show up until run time:

 
position = "2"               #string, where a number was intended
position = positoin + 4      #position is misspelled, evaluates to 0
puts "The position is:" + 
      position.to_string     #The method should be to_s  

Bugs like these are trivial to solve if a compiler catches them, but if you've got to rely on an interpreter, they are much more difficult to manage. To deal with these kinds of subtle errors, users of dynamic languages have long relied on automated testing. Dynamic languages in general and integrated environments in them specifically have significant advantages over other languages when it comes to testing:

• The languages are more concise. Testing is basically scripting, and many of the best scripting languages are dynamically typed.
  
  
• Integrated environments allow assumptions that make integrating testing much easier, and potentially more powerful. You'll see some examples in the Rails environment.
  
  
• Dynamic languages allow looser coupling, making some testing patterns easier to implement.

Now that you know why dynamic language developers are passionate about testing, it's time to build a real application that needs some real tests.

Back to top

Building a quick Rails application

To get going quickly, I'll scaffold together a Rails application for keeping a database of mountain biking trails. I'll pull together a few tests for the model. If you want to code along, all you need is a database engine (I'll use MySQL) and Ruby on Rails version 1.1 or newer (see Resources). The first step is to create a Rails project. From your command prompt, type the rails trails command, shown in Listing 2 along with the command's results:

 
> rails trails
create  
create  app/controllers
create  app/helpers
create  app/models
create  app/views/layouts

...partial results deleted...

create  test/fixtures
create  test/functional
create  test/integration
create  test/mocks/development
create  test/mocks/test
create  test/unit
create  test/test_helper.rb

...partial results deleted...

create  config/environment.rb
create  config/environments/production.rb
create  config/environments/development.rb
create  config/environments/test.rb

...partial results deleted...

create  log/server.log
create  log/production.log
create  log/development.log
create  log/test.log

Rails has done nothing but generate your empty project, but you can already see that it's working for you. Among the directories created in Listing 2 are:

• Application directories with subdirectories for model, view, and controller
• Test directories for unit, functional, and integration tests
• An environment explicitly created for testing
• A log for the results of your test cases

Because Rails is an integrated environment, it can make assumptions about the best ways to structure and organize the testing framework. Rails can also generate default test cases for you, as you'll see.

Now you'll create a database table through a migration and then use the database table to create a new database. Change into the trails directory by typing cd trails. Next, generate a model and migration, as shown in Listing 3:

 
> script/generate model Trail
          exists  app/models/
          exists  test/unit/
          exists  test/fixtures/
          create  app/models/trail.rb
          create  test/unit/trail_test.rb
          create  test/fixtures/trails.yml
          create  db/migrate
          create  db/migrate/001_create_trails.rb

Note that if you're using Windows, you must preface your commands with Ruby, so your command would be ruby script/generate model Trail.

As you can see in Listing 3, the Rails environment creates not just your model, but also migrations, a test case, and a test fixture. You'll see more of the fixture and test in a moment. Migrations let Rails developers deal with the inevitable changes in database tables throughout the development process (see Crossing borders: Exploring Active Record). Edit your migration (in 001_create_trails.rb) to add the columns you'll need, as shown in Listing 4:

 
class CreateTrails < ActiveRecord::Migration
   def self.up
      create_table :trails do |t|
         t.column :name, :string
         t.column :description, :text
         t.column :difficulty, :string
   end
 end

   def self.down
      drop_table :trails
   end
end

You need to create and configure two databases: trails_test and trails_development. You'd need a third, called trails_production, should you want to push this code into production, but you can skip that step for now. Use your database manager to create your databases. I'll use MySQL:

mysql> create database trails_development;
Query OK, 1 row affected (0.00 sec)

mysql> create database trails_test;
Query OK, 1 row affected (0.00 sec)

Then edit the configuration in config/database.yml to reflect your database preference. Mine looks like this:

development:
   adapter: mysql
   database: trails_development
   username: root
   password:
   host: localhost


test:
   adapter: mysql
   database: trails_test
   username: root
   password:
   host: localhost

Now, you can run your migration and then scaffold together the rest of the application:

> rake migrate

...results deleted...
    
> script/generate scaffold Trail Trails
...results deleted...

    create  app/views/trails

    ...results deleted...

    create  app/views/trails/_form.rhtml
    create  app/views/trails/list.rhtml
    create  app/views/trails/show.rhtml
    create  app/views/trails/new.rhtml
    create  app/views/trails/edit.rhtml
    create  app/controllers/trails_controller.rb

    create  test/functional/trails_controller_test.rb

    ...results deleted...
 

Once again, note the test cases that Rails has created for you. The framework has generated not only the views and controller for a simple little application, but also a functional test to help you test the user interface.

Back to top

Unit testing your Rails application

It's time to run some tests. Look at the first test, which is already written for you, in test/unit/trail_test.rb:

require File.dirname(__FILE__) + '/../test_helper'

class TrailTest < Test::Unit::TestCase
   fixtures :trails

   # Replace this with your real tests.
   def test_truth
      assert true
   end
end

Granted, that's not much of a test case, but you can see how to structure your test code, and the template for your test cases is already in place. Run the test, as shown (along with part of the result) in Listing 9:

> ruby test/unit/trail_test.rb 
    Loaded suite test/unit/trail_test
    Started
    EE
    Finished in 0.027314 seconds.

      1) Error:
    test_truth(TrailTest):
    ActiveRecord::StatementInvalid: Mysql::Error: #42S02Table 
      'trails_test.trails' doesn't exist: DELETE FROM trails

...results deleted...

The test case failed, but take a look at the output. The first line executes the test. The third line with EE shows the results of the tests. If a test case passes, you get a "." character. If the test case produces an error, you see an E. If any assertion is not true, you see an F. Next, you see that all tests that you requested finished and the time it took to complete them. Finally, you see the detailed reason for each failure. In this case, the table doesn't exist, which makes sense, because you didn't create any tables in the test database. Remedy that error by cloning the development schema to the test environment and then running the test again, as shown in Listing 10:

> rake clone_schema_to_test          (in /Users/batate/rails/trails)
> ruby test/unit/trail_test.rb 
    Loaded suite test/unit/trail_test
    Started
    .
    Finished in 0.038578 seconds.

    1 tests, 1 assertions, 0 failures, 0 errors

That's better. But the test is too simple, so it's time to build a real test case. Add a new test case below test_truth, as shown in Listing 11:

    def test_truth
      assert true
    end

    def test_new
      trails = Trail.find_all
      Trail.new do |trail|
        trail.name = "Barton Creek"
        trail.description = "A little water in the Spring. You'll get wet."
        trail.difficulty = "medium"
        trail.save
      end
      bc = Trail.find_by_name("Barton Creek")
      assert_equal "medium", bc.difficulty
      assert_equal trails.size + 1, Trail.find_all.size

    end

This code is marvelously compact. You needed to type only the code to manipulate the persistent model, along with a couple of assertions. This economy of effort is what makes scripting languages so popular in other environments. Testing, too, is a place that calls for economy of effort.

Now you can run the test case, and you'll see your two new assertions show up in your test report. With Ruby, just save your test case and compile. Listing 12 shows the results of the test run:

> ruby test/unit/trail_test.rb 
    Loaded suite test/unit/trail_test
    Started
    .
    Finished in 0.038578 seconds.

    1 tests, 1 assertions, 0 failures, 0 errors
    bruce-tates-computer:~/rails/trails batate$ ruby test/unit/trail_test.rb 
    Loaded suite test/unit/trail_test
    Started
    ..
    Finished in 0.182043 seconds.

    2 tests, 3 assertions, 0 failures, 0 errors

Fixtures and rollback

Java mock objects Java developers often use mock objects, rather than actual database code, to solve the problems that beset testing database-backed code. Mock objects are hard to set up, often difficult to understand, and not as likely to provide a good understanding of code working in the context of a database. Ruby on Rails enables a different approach.

Three problems mar testing with database-backed code. They all relate to two characteristics: performance and repeatability. Performance of database calls, when compared to in-memory operations, is very slow. When tests take too long to run, you probably won't run them. Another problem is the impact of one test case on another. Because database calls are by nature persistent, it's harder to isolate one test case from database changes in another. The final problem is a combination of the first two. When you add the burden of setup and teardown -- adding records, running tests, and deleting those records for each new test case -- to make database test cases repeatable, the overhead can get overwhelming. This overhead can easily dwarf the overhead for the test case.

Ruby on Rails helps to solve these problems with fixtures and with transaction rollback. In Rails, a fixture is a file containing data for your test cases. When you created this simple application, you created a development database and a testing database. The development database is natural; you don't want your production code and your development environment to share the same database. The testing database is important for a different reason. Each test loads the test data in your fixtures at the beginning of a test case. Then, your test case makes changes to the database and tests the results of those changes. Finally, Rails rolls those changes back to return the database to the state that existed before the test method ran.

Now you'll make a test fixture and write a test against it. Edit the test/fixtures/trails.yml file to add a record, as shown in Listing 13:

    first:
      id: 1
      name: "Emma Long"
      description: "A real bike breaker."
      difficulty: "hard"
    another:
      id: 2
      name: "Bear Creek"
      description: "Too many downed trees."
      difficulty: "easy"

Listing 13 uses a language called YAML, which describes structured data (see Resources). It's sensitive to white spaces, so you should use spaces instead of tabs and type the items exactly as you see them, making sure you strip all trailing spaces. 

Also, add this test case to trails_test.rb:

    def test_find
      assert_equal "Emma Long", Trail.find(1).name
      assert_equal "easy", Trail.find(2).difficulty
    end

Once again, you can run the tests with five passing assertions. You can also refer to each of the fixtures by name, if you wish. For example, to create an object from the fixture called first, you use the Ruby code trails[:first]. Having fixtures available to all test cases, or just the ones that need them, dramatically simplifies the code that you need to create or destroy database data.

Back to top

Testing in Java programming

Knowing how testing happens in other languages can change the way you do testing on the Java platform. In particular, using one or more of these ideas can make a significant, and immediate, impact on your testing:

• You can add test-case generation to any of your existing code generation. Ruby on Rails gets tremendous leverage by creating some simple test cases by default. You can too.
  
  
• You can use the transaction-rollback technique to make your database-backed tests run faster. The Spring framework has some existing interceptors that make this technique easy to use.
  
  
• You can actually use dynamic languages to drive your testing. Jython, Ruby, and Groovy are three real possibilities.

If you think you'd like to use other languages to test, you can use one of the JVM languages, such as JRuby (see Resources). JRuby isn't advanced enough yet to run Ruby on Rails, but it's an outstanding testing platform for Java applications. For just a taste, the JRuby developer Charles O'Nutter provided this example for testing an EJB:

    require 'test/unit'
    require 'java'

    include_class "my.pkg.EJBHomeFactory"

    class TestMyBean < Test::Unit::TestCase 
      def test_finder
        wh = EJBHomeFactory.widget_home
        w = wh.find_by_color("blue")
        assert_not_nil(w)
      end

      def test_widget
        wh = EJBHomeFactory.widget_home
        w = wh.find_by_name ("superWidget")

        assert_equal("blue", w.color)
        assert_equal(14, w.id)
      end
    end

You can see that it's actually quite trivial to write test cases in Ruby that exercise Java code. Using this example, the Ruby code finds an EJB component and makes some assertions about the bean returned by the user. The test case is certainly simpler than most Java tests, and using Ruby for this kind of thing is a way to be very productive, very quickly. I've seen similar strategies for Jython or Groovy (see Resources).

Part 2 takes a deeper look into Rails testing, including code to run higher-level tests called functional and integration tests.

Resources

• Beyond Java (O'Reilly, 2005): The author's book about the Java language's rise and plateau and the technologies that could challenge the Java platform in some niches.
  
  
• Java To Ruby: Things Your Manager Should Know (Pragmatic Bookshelf, 2006): The author's book about when and where it makes sense to make a switch from Java programming to Ruby on Rails, and how to make it.
  
  
• Programming Ruby (Dave Thomas et al., Pragmatic Bookshelf, 2005): A popular book on programming Ruby.
  
  
• "Running your Rails App Headless" (Mike Clark's Weblog, April 2006): Mike Clark shows the integration testing framework for Ruby on Rails.
  
  
• YAML fixtures: Learn more about Rails fixtures.
  
  
• YAML: A machine-parsable data format optimized for data serialization, configuration settings, log files, Internet messaging, and filtering.
  
  
• Demystifying Extreme Programming (developerWorks): XP is the most popular agile approach to software development.
  
  
• al.lang.jre (developerWorks): This series introduces alternate languages for the Java Runtime Environment and includes articles on JRuby, Jython, and Groovy.
  
  
• "Practically Groovy: Unit test your Java code faster with Groovy" (Andrew Glover, developerWorks, November 2004): Learn a simple strategy for unit testing Java code with Groovy and JUnit.
  
  
• "Introduction to the Spring framework" (Rod Johnson, TheServerSide, May 2005): Don't miss the part of this article that talks about rolling back a transaction at the successful completion of a test case.
  
  

• Ruby on Rails: Download the open source Ruby on Rails Web framework.
  
  
• Ruby: Get Ruby from the project Web site.
  
  
• JUnit: The Java testing framework that started the automated testing craze for the Java platform.
  
  
• TestNG: A next-generation testing framework for Java development.
  
  
• JRuby: A Ruby implementation that runs in the JVM.
  
  
• Selenium: An integration testing framework for Web applications.
  
  
• JUnitPerf: A collection of JUnit test decorators for measuring performance and scalability within existing JUnit tests.
  
  

• Improve Your Code Quality discussion forum: For questions related to testing and code quality, this discussion forum, hosted by Andrew Glover, is a must.

About the author

		Bruce Tate is a father, mountain biker, and kayaker in Austin, Texas. He's the author of three best-selling Java books, including the Jolt winner Better, Faster, Lighter Java. He recently released Beyond Java.. He spent 13 years at IBM and is now the founder of the RapidRed consultancy, where he specializes in lightweight development strategies and architectures based on Java technology and Ruby on Rails.

Rate this page

Please take a moment to complete this form to help us better serve you. Did the information help you to achieve your goal? Yes No Don't know   Please provide us with comments to help improve this page:   How useful is the information? 1 2 3 4 5 Not useful Extremely useful

Share this.... Digg this story del.icio.us Slashdot it!

Back to top