Getting started with Rational Robot automation framework support (RRAFS)

Document options Document options requiring JavaScript are not displayed Rate this page Help us improve this content

Level: Introductory

Jirong HuDevelopment Bank of Singapore

15 Jun 2004

Rational Robot automation framework support (RRAFS) is one of the most popular data/keyword-driven test automation frameworks available today. This article will help you get started with the framework quickly and apply it to testing Web applications.

Editor's note: This article applies to IBM® Rational® Robot version 2001 and later, and RRAFS version 2003.08.27.00 and later.

Rational Robot Automation Framework Support (RRAFS), initially developed by Carl Nagle for the SAS Institute, is one of the most popular data/keyword-driven test automation frameworks available today. Having done a few small test automation projects and a lot of online reading, we adopted RRAFS just before starting a large testing project. We knew that with the additional layer of abstraction it offers, it brings test automation within reach for testers who aren't programmers and thus was perfect for our team of nontechnical testers. Still, it took us a few weeks to really become productive with RRAFS. One of the reasons is that because it's still new, it lacks a user guide, samples, and other resources. This article will fill the gap to help you get started with the framework quickly and apply it to testing Web applications. You can visit the main support site for RRAFS for more information.

About Test Automation Frameworks

What's a test automation framework, and why do you need one? Can't you just do a record-and-playback to create test automation scripts with Rational Robot? Isn't the tool good enough without a framework?

Carl Nagle defines a testing framework as "the collection of tools and processes available to develop, execute, and analyze software tests." I recommend reading Carl Nagle's paper "Test Automation Frameworks" as well as "Totally Data-Driven Automation Testing" by Keith Zambelich to understand why you need a totally data/keyword-driven automation framework. Another good starting point is the article "Choosing a Test Automation Framework," where Mike Kelly compares five different types of frameworks. But the best way to start is just to dive in and try it out, which this article will help you do.

Some of the basic problems with a simple record-and-playback automation strategy are these:

• The tester has to be a programmer, which is rare in real life.
• The test scripts break too easily since they're hard-coded.
• All testers have to be tools experts.

On the other hand, using RRAFS offers these main advantages:

• It provides an additional layer of abstraction.
• It's completely data-driven.
• It supports different tools.

Still, for many reasons, it may not be easy to convince your management to adopt a test automation framework. When I asked in Rational Discussions why we should adopt such a framework, Steve Dryja gave me this answer: "Which would you prefer -- developing your own set of libraries to do what has already been created for us or copying the free libraries to your environment and start writing test plans/test cases, which skips the coding step and puts you much closer to having your test plans automated. This will save us money and time because someone does not have to code everything. The test team can spend their time writing test plans."

Back to top

Installing RRAFS

RRAFS is currently hosted on SourceForge and is an implementation of the project named Software Automation Framework Support. Installing it is pretty straightforward.

First, download all the files for the stable release from the rrafs engine file package. At the time of this writing, the stable release is RRAFS2003.08.27.00 and these are the files to download:

• ReleaseNotes2003.08.27.00.txt
• RRAFS2003.08.27.00.ZIP
• RRAFS2003.08.27.DOC.ZIP
• RRAFS2003.08.27.XML.ZIP

A complete installation guide, named DDEngineSetup.htm, can be found in the RRAFS2003.08.27.DOC.ZIP file. It's available online as well.

If this is your first time installing RRAFS and you have Robot version 2001 or later, there are basically just four steps:

1. Open Robot and click Tools > GUI Playback Options > Error Recovery tab and set both "On script command failure" and "On verification point failure" to "Continue execution," as shown in Figure 1.

Figure 1: Our settings on the Error Recovery tab

2. Unzip all files from the downloaded RRAFS2003.08.27.00.ZIP to Robot's SQABasic path, defined on the Preferences tab in the General Options window (see Figure 2) -- for example, C:\Program Files\Rational\Rational Test\sqabas32.

Figure 2: Determining Robot's SQABasic path

3. Double-click the Setup.vbs found in the SQABasic path and accept all default settings. It will register all the new DLLs for the framework.
4. Create the project from Rational Administrator before you start, with a directory structure in your project directory like this one:

   D:\Repository\RRAFS\Datapool\Bench
D:\Repository\RRAFS\Datapool\Dif
D:\Repository\RRAFS\Datapool\Logs
D:\Repository\RRAFS\Datapool\Runtime (OPTIONAL)
D:\Repository\RRAFS\Datapool\Test

   D:\Repository\RRAFS\ is our project repository to run the sample in this article. You can use your own project directory instead if you want.

The latest release and patches are available in the rrafs engine file package. Follow the instructions included to install the patches as needed.

Back to top

Trying RRAFS on the ClassicsC Sample

After you've installed RRAFS, the best way to become familiar with it is to run it against the ClassicsC sample. If you're working with a Web application, you might just want to pass on this and go to the next section, "Trying RRAFS on the JPetStore Sample Web Application."

First you need to install the original Visual Basic ClassicsC sample from Rational on your machine if it isn't there already. From TeamTest, click Rational Test > Set Up Rational Test Samples (see Figure 3). Accept all default settings to install all the samples.

Figure 3: Installing the ClassicsC sample from TeamTest

Now you can run the ClassicsC RRAFS demo, contained in the ClassicsC_V2001.zip file included in RRAFS2003.08.27.DOC.ZIP. Detailed instructions on how to install this demo can be found in the ClassicsC_v2001_Readme.txt file included in the ClassicsC_V2001.zip file. Basically, you just do this:

1. Copy the files ClassicsC_HIGH.xls, ClassicsC_MAP.xls, and ClassicsC_STEPS.xls to the D:\Repository\RRAFS\Datapool\ folder.
2. Copy ClassicsMainMenu.mnu to the D:\Repository\RRAFS\Datapool\Bench\ folder.
3. Copy CycleDriverTest.rec, ExportXLTables.rec, and ExitMainWin.rec to the script directory D:\Repository\RRAFS\TestDatastore\DefaultTestScriptDatastore\TMS_Scripts\.
4. Compile all the scripts.

Note that to make the scripts you copied in step 3 visible to Robot, so you can see them when you click File > Open, you need to create a new GUI script for each with the same name -- for example, CycleDriverTest -- in Robot. Once you click the OK button, the original script's content will be loaded into the new script.

To run the RRAFS ClassicsC demo and see the action of the framework, run the CycleDriverTest script from Robot. Spend some time on the demo to understand the logic behind it, starting with the three Excel files you've copied.

Back to top

Trying RRAFS on the JPetStore Sample Web Application

Carl's RRAFS documentation comes with the above-mentioned ClassicsC demo and a NotePad demo, but no Web application demo. This made it hard for me to learn RRAFS at first because there are some differences in how you use the framework for testing a Web application. Some of the differences are as follows:

• A Web application usually uses StartWebBrowser instead of LaunchApplication.
• A Web application is usually slower than a client-server application, so you have to take additional steps to make the test script robust.
• A Web application uses HTML pages.

It's hard to find a good Web application sample for demo purposes, mainly because any real-life Web site could change in the near future (which is another good reason to adopt the framework), and that would break any demo based on it. I finally chose the famous JPetStore application as the basis of our demo for this article, for these reasons:

• It's the so-called blueprint application for a J2EE application and the business logic won't change in the near future.
• The current JPetStore implementation is based on Apache Struts, a de facto Web tier framework in the J2EE world. I hope RRAFS will be as popular as Struts someday.

You can access the JPetStore sample application online.

Running the JPetStore Demo

Assuming you've already installed the RRAFS framework, follow these steps to run the JPetStore demo:

1. Download Getting Started RRAFS.zip.
2. Copy the files JPetStore_HIGH.xls, JPetStore_MAP.xls, and JPetStore_STEPS.xls from the Datapool folder to your D:\Repository\RRAFS\Datapool\ folder. (You can safely extract the whole directory of this folder to your test repository root directory.)
3. Create a new GUI script named JPetStore_CycleDriver and copy into it the contents of the file JPetStore_CycleDriver.rec.
4. Create a new GUI script named JPetStore_ExportXLTables and copy into it the contents of the file JPetStore_ExportXLTables.rec.
5. Optionally, you can create a script containing the file Launch_ProcessContainer.rec (more about why you'd want to do this later).
6. Compile all the scripts and run the JPetStore_CycleDriver script.
7. View the log files at D:\Repository\RRAFS\Datapool\Logs\.

How It Works

The logic of navigating a Web application can be summarized as simply as this:

1. Go to a Web page.
2. Find a particular component.
3. Act on that component.

All interactions with a Web application repeat this logic and nothing more. RRAFS implements this logic in the framework engine. We just need to pass in the data for these three steps -- for example, the URL of the page or the caption of a Web page for the first two steps, the recognition string of the component for the second step, and the action for the third step. During test execution, the framework reads this data from the Excel files that we provide and follows the test steps specified there.

We'll look now at the role of each of the five files you copied -- JPetStore_CycleDriver.rec, JPetStore_ExportXLTables.rec, JPetStore_MAP.xls, JPetStore_HIGH.xls, and JPetStore_STEPS.xls. The diagram in Figure 4 shows the workflow and our scripts' interaction with the framework.

Figure 4: The workflow of our JPetStore demo

(click here to enlarge)

JPetStore_CycleDriver.rec

JPetStore_CycleDriver.rec is the entry point for the test. The script starts by creating the log files, then parsing and exporting the data from the Excel sheets to text files. Then it calls the framework engine to execute the test and finally terminates the application.

As you can see from the code, the location of log files, the name of the script to export the data from the Excel sheets, and the name of the regression table are defined here.

sTemp = SQAGetDir(SQA_DIR_PROJECT)+"Datapool\Logs\JPetStore_SuiteLog.txt"
CallScript "JPetStore_ExportXLTables"
'launch the DDE Engine with the cycle test
CDCycleDriver "JPetStore_Regression.cdd", "", CycleLog, SuiteLog, MainLog,
CDCycleDrivenMode

JPetStore_ExportXLTable.rec

Running JPetStore_ExportXLTable.rec simply exports the test data from the Excel sheets and creates text files expected by the test engine. These are all the files with the extensions .CDD, .STD, .SDD, and .MAP. By default they're placed in D:\Repository\RRAFS\Datapool.

Notice that the names of the Excel sheets and the map file are given here.

'XLS input files
Const HighLevelTables="JPetStore_HIGH.XLS"
Const LowLevelTables ="JPetStore_STEPS.XLS"
Const ApplicationMap ="JPetStore_MAP.XLS"

'Single MAP output file
Const MapOUT = "JPetStore.MAP"

Next let's take a close look at these Excel sheets.

JPetStore_MAP.xls

The application map file contains all the information Robot needs for the first two steps of navigating a Web application, the location of the Web page and component. It defines a so-called object recognition string (the same string you need to provide to the SQAGetProperty method) for each object in the application under test.

Most of the time, Robot's powerful Process Container will do all the hard work of finding these object recognition strings for you. In this demo, I used the settings shown in Figure 5 to create the whole application map with Process Container. I'll say more about using the Process Container later.

Figure 5: The Process Container settings I used

The resulting ApplicationMap_JPetStore.map file contains all the components Process Container can find in each page. Most of the time, we don't need all of them, so we copy the components we're interested in to the Excel sheet and give them a meaningful name. For example, only two entries are necessary for the first page:

[FirstPage] FirstPage="Type=Window;Caption={JPetStore*}" EnterLink=Type=HTMLLink;HTMLText=Enter theStore

JPetStore_HIGH.xls

This sheet is the data table for the suite driver. It contains two sections, a regression table (in our demo, JPetStore_Regression.CDD) and multiple test suite tables (in our demo, only PurchaseFish.STD). We use the regression table to define all the test suites we want to run for this regression. The suite table contains the names of the lower-level step tables for each test suite.

Here's our regression table, JPetStore_Regression.CDD (RT = record type):

RT	Suites	Arg	Description
T	PurchaseFish

And here's our test suite table, PurchaseFish.STD:

RT	Steps	Sep	Arg	Arg
T	AccessStore
T	PurchaseFish		^number="3"
T	LeaveStore

AccessStore, PurchaseFish, and LeaveStore in the suite table refer to the step tables in the JPetStore_STEPS.xsl sheet. They're exported as AccessStore.SDD, PurchaseFish.SDD, and LeaveStore.SDD by JPetStore_ExportXLTables.rec at runtime. Note that these names must be exactly the same in the suite table and the step table and can't have duplication since the framework finds them by name. You can create hyperlinks in Excel to link all of them together for easy navigation.

Here's a short description of the record types used in the far left column, here and in the step tables:

• B = BLOCKID -- Start of a named block (not used at this time).
• C = DRIVER COMMAND -- Command to driver to perform some function other than a test step.
• S = SKIPPED TEST -- Record that's skipped; a user-defined comment is logged as to why.
• T = TEST STEP -- Test step to perform.

Refer to the online RRAFS Reference for a more detailed description.

JPetStore_STEPS.xls

This file contains all the detailed step tables, which define the lowest level of commands to execute the test case. Here are the tables on the AccessStore.SDD worksheet:

These step tables define the following sequential actions:

1. Set the application map to JPetStore.map.
2. Start the Web browser and load the page at http://www.jwebhosting.net/jpetstore/.
3. Maximize the browser window.
4. Click the Enter Store link.

Back to top

Getting Started with a New Project

Before you start using RRAFS on a testing project of your own, it helps to read the documentation. Carl Nagle's "Test Automation Frameworks" should give you a very good idea of RRAFS. After that, you should read CreateAppMap.htm, ProcessContainer.htm, and Using DDVariables.htm to understand these major components of RRAFS. All these documents can be found in RRAFS2003.08.27.DOC.ZIP or in the online RRAFS Reference. Browse through the reference document and look at component functions, driver commands, and so forth.

Then you'll be ready to use RRAFS. I'd suggest that you proceed as follows:

1. Print your screens.
2. Copy the tables and scripts from the JPetStore demo.
3. Create the application map.
4. Create the HIGH and STEPS tables.

I'll go over each of these steps and include helpful hints and tips where appropriate.

Print Your Screens

The first thing you should do in a real project is to print every screen in the application under test. Then do the following:

1. Give a name to each of these pages.
2. Identify the components you need for testing in each screen.
3. Define the page flow in step tables.

There will be many times you'll need to refer back to these screen shots. When you have them in hand, you won't need to access the real application all the time, something that may not always be practical. For example, a test case will result in a change of test data so you'll need help from the host team to restore the data after running. There'll be a lot of system downtime during your development as well.

Copy the Tables and Scripts from the JPetStore Demo

If you're working with a Web application, I suggest that you copy the three Excel files from the JPetStore demo and then delete the demo data and replace it with your own. (Sometimes the changes aren't reflected after you update these .xls files. When this occurs, try deleting all the files generated from these at runtime. If that still doesn't fix your problem, restart Robot.)

Then copy CycleDriver.rec, ExportXLTables.rec, and Launch_ProcessContainer.rec into new Robot GUI scripts and modify them as necessary. You'll be using the Launch_ProcessContainer script in the next step. You'll run the CycleDriver script in order to do your testing, and it will call ExportXLTables.rec.

Create the Application Map

To create the application map, you need to master using the Process Container efficiently. Follow these steps:

1. Launch the Process Container by running the Launch_ProcessContainer script within Robot.
2. Run the Process Container for every page in your Web application.
3. Give each page a unique name in the Window\Object Name field (see Figure 5).
4. Always make sure to check Append AppMap and give the application map file a name -- for example, ApplicationMap_JPetStore.map.
5. Always make sure Remove Parent Info is not checked.

Remember to tab out the control you've changed. Here are some additional tips about running the Process Container:

• If you don't check Append AppMap, the Process Container will generate only the map file for each page you run the Process Container on -- for example, HomePageObj.txt. Inside the file you'll find a lot of 1015 errors. This is due to a known Robot defect and basically means there's no child anymore under this component. When Append AppMap is checked, the Process Container will also generate a complete application map, which is easier to read and use.
• You can safely use the wildcard character (*) in the Window Recognition Method field but have to put the full HTML title in the Object Recognition Method field for HTML pages.
• If you have a Java applet in the HTML page, you need to run the Process Container two times to get the complete map (see also the tip below under "Fix the JavaEnabler Problem"). Choose Web Client to get the recognition string for HTML components and choose Java Client to get the recognition string for Java controls.

When the above process is finished, you should have a complete application map in hand -- for example, ApplicationMap_JPetStore.map. The Process Container will at the same time create a map file in \Datapool\ for each page named with a Window\Object Name -- for example, JPetStore_HomePageObj.txt.

Analyze the application map file and copy the recognition strings for those components you're interested in to the Excel sheet. One tip about this: Sometimes you aren't able to map the controls on the screen to the recognition string generated by the Process Container. For example, in the TigerShark page, we have:

HTMLLink12="Type=HTMLDocument;HTMLTitle=JPetStore
   Demo;\;Type=HTMLLink;Index=12"
HTMLImage17="Type=HTMLDocument;HTMLTitle=JPetStore 
   Demo;\;Type=HTMLImage;Index=17"
HTMLTableCell15="Type=HTMLDocument;HTMLTitle=JPetStore 
   Demo;\;Type=HTMLTableCell;Index=15"
HTMLLink13="Type=HTMLDocument;HTMLTitle=JPetStore 
   Demo;\;Type=HTMLLink;Index=13"
HTMLImage18="Type=HTMLDocument;HTMLTitle=JPetStore
   Demo;\;Type=HTMLImage;Index=18"

Looking at that, we don't know which one is the "Add to Cart" image link that we want to click since no meaningful names have been given to the HTML object. If this happens, you can use Robot's Object Data VP or Inspector (Tools > Inspector) to help.

When you've finished this whole process, you should have a complete application map in the format of an Excel spreadsheet.

Create the HIGH and STEPS Tables

The HIGH table is where you organize all your test suites for this project, as you would in TestManager. You can define different suites for different test cycles. The STEPS table is where you define the detailed test steps according to your test plan. You'll need to consult the online RRAFS Reference to construct these steps.

Back to top

More Tips

I'll leave you with three more tips that can save you time when you start using RRAFS.

Disable Launching of TestManager

You don't really need TestManager when using RRAFS. I never think TestManager is useful in GUI testing anyway, except when I'm testing different platforms. To disable the launching of TestManager from Robot, click Tools > GUI Playback Options > Log and uncheck the "View log after playback" entry, as shown in Figure 6. Don't disable the log completely since some of the commands still depend on that.

Figure 6: Disabling launching of TestManager

Fix the JavaEnabler Problem

For Robot to be able to identify a Java applet in an HTML page, you need to run the JavaEnabler included in Robot (see Figure 7). You can choose to install it from the Web to get the latest version of JavaEnabler.

Figure 7: Running JavaEnabler

The problem is that even when I ran JavaEnabler, Robot still wasn't able to recognize the Java applet in one of the pages I was testing, showing it as a JavaWindow. I was using Internet Explorer 6 and Windows XP Professional. I finally had to install the latest Sun JDK to fix the problem. You can choose the JVM for your IE browser in Robot by clicking Tools > Internet Options > Advanced tab and under Java (Sun) checking "Use Java2 v1.4.2 for <applet>" (see Figure 8). Consult your development team to find out the version of JVM required for the application under test.

Figure 8: Choosing the JVM for your IE browser

To verify whether this worked, you can run the Java version of the Rational test sample TryIt. Robot should be able to recognize the lower-level Java controls such as Java EditBox. If Robot shows only a JavaWindow when you do a simple Object Data VP on the EditBox, it's not working. It's also not working when you see the following error when you run the Process Container:

Call Robot Scripts from RRAFS

A nice feature of RRAFS is that you can call a Robot script directly from the framework, giving you the option of bypassing the framework at any time should that be necessary. Instead of specifying a record type in the step table, you enter the Robot script name in the first column (or using CallScript), and the script will be called by the framework at that point.

For example, before I fixed the JavaEnabler problem, I had to call the Robot script for my login window (Login_Pin.rec) in our Internet banking application, which uses a Java applet for the PIN box. Here was the step table in that case (remember that record type S means SKIPPED TEST):

Back to top

You're on Your Way

By providing an additional abstraction layer, RRAFS shields test automators from having to do tools scripting and thus brings test automation within reach of normal testers. In this article you've had a chance to see how RRAFS works in the case of testing a Web application, and you've learned how to get started with using it on your own testing project. Once you've applied it to your own projects, you'll probably never want to go back to using only record-and-playback scripting with Robot.

Back to top

References and Other Resources

• "Choosing a Test Automation Framework" by Mike Kelly
• "Totally Data-Driven Automation Testing" by Keith Zambelich (Automated Testing Specialists, Inc., Web site)
• "Test Automation Frameworks" by Carl Nagle (SAFSDEV Web site)
• RRAFS Discussions
• RRAFS Reference

About the author

		Jirong Hu is the software process lead at the Advanced Delivery Center, Development Bank of Singapore. He has been working in J2EE development with WebSphere for years. Currently he is working in the area of SCM and test automation with Rational tools. You can reach Jirong at hujirong888@yahoo.com.

Rate this page

Please take a moment to complete this form to help us better serve you. Did the information help you to achieve your goal? Yes No Don't know   Please provide us with comments to help improve this page:   How useful is the information? 1 2 3 4 5 Not useful Extremely useful

Share this.... Digg this story del.icio.us Slashdot it!

Back to top