 | Level: Intermediate Vaibhav Telang (vaibhav.telang@in.ibm.com), Software Engineer, IBM India Software Labs, Gurgaon, India
13 Sep 2005
Updated 10 Nov 2005 Learn how the IBM Lotus System Test Reliability team used an IBM Rational TestManager command line execution approach for reliability testing of IBM Workplace Collaborative Services 2.5. This article explains how the IBM Lotus System Test Reliability team adapted an IBM Rational TestManager suite execution strategy to ensure that IBM Workplace Collaboration Services 2.5 was adequately tested for reliability, in spite of certain tool challenges.
By sharing this technique, we hope to help automation engineers who plan to schedule and run Rational TestManager suites automatically, in a sequential manner and without human intervention. Automation Teams that are good candidates for using this technique are teams that want to simulate actual usage patterns by executing Rational TestManager suites over longer durations, and also want to resurrect the runs periodically. This article assumes that you are an experienced software tester, ideally with some familiarity with IBM Rational test tools.
Workplace Collaboration Services is an innovative product from the IBM Workplace family of products. It provides an integrated collaborative environment with a wide range of features such as e-mail, calendaring and scheduling, presence awareness and instant messaging, learning, team spaces, Web conferencing, and document and Web content management. The System Test Reliability testing team used Rational TestManager and IBM® Rational® Robot to record VU scripts, and to create and execute Rational TestManager suites. Rational TestManager is the central console for test activity management, execution, and reporting. Built for extensibility, it supports everything from purely manual test approaches to various automated paradigms, including unit testing, functional regression testing, and performance testing.
The challenge
Reliability execution requires testers to emulate actual usage scenarios. Actual usage patterns of a collaborative software product like Workplace Collaborative Services requires a detailed operational profile with a collaborative mix covering all the product components. This operational profile needs to be translated into a Rational TestManager suite which can be executed for extended durations (24 - 96 hours) to test the product for reliability. The team was required to build usage patterns depicting different work hours for users in various time zones.
To simulate this scenario, we were required to schedule Rational TestManager suite execution at particular time without manual intervention. There is currently no mechanism available in Rational TestManager to schedule test suite execution. Also, there was no available mechanism to restart a terminated virtual user in the middle of suite execution. Both these challenges required the team to adopt a different strategy then the default provided by IBM Rational TestManager.
The solution
To overcome these challenges, the Reliability team adopted a staggered execution strategy. The longer duration executions (24-96 hours) were divided into smaller executions. These small duration executions were then kicked off using a command line approach with a batch file. This approach allowed resurrecting virtual users whenever a new execution started, and also enabled automatic workload execution kickoff by using Microsoft® Windows® Task Scheduler.
The folllowing sections will explain how we set up the Rational TestManager environment and used Windows Task Scheduler for scheduling a batch file to kick off Rational TestManager suites for reliability testing. Rational TestManager allows suites to be executed from its GUI or from a command line. In scenarios where you need to schedule an execution at a particular time -- or in cases where you want to execute Rational TestManager suites sequentially, without human intervention -- you should consider the method of running suites from a command line using a batch file.
The following syntax shows you how to run a suite from the command line. If an argument value contains spaces, enclose it in quotation marks.
Listing 1. Running a suite from the command line
rtmanager.exe suitename /runsuite /user userid [/password password]
/project .rsp-path [/computers [Local]; [computer-1; computer-2; ... computer-n]
[/computerlists computerlists] /build buildname /logfolder logfoldername /log logname
[/overwritelog] [/numusers nnn] [/ignoreconfiguredtestcases] [/close]
|
Here is a description of the syntax elements:
-
suitename /runsuite: The name of the suite to run. It runs the suite referenced in suitename.
-
/user userid: The user name for logging on to TestManager.
-
/password password: An optional password for logging on to TestManager. Omit this option if there is no password.
-
/project .rsp-path: The full path to the project .rsp file.
-
/computers [Local]; [computer-1; computer-2;…computer-n]: The names of the computers to use when running the suite. Local is the computer on which TestManager is running. This option is ignored if the specified suite indicates the computers on which to run it. If the suite does not indicate the computers or computer lists to use, this option is required.
-
/computerlists [computerlist-1; computerlist-2,… computerlist-n]: The computer lists to use when running the suite. This option is ignored if the specified suite indicates which computers to use to run it, or if you specified computers in the /computers option. Note: If you specify /computerlists Local then the suite will be run for only one virtual user. If you want to run the suite for multiple users then remove this option. Using the /computers switch causes Rational TestManager to decide that there is only one computer available for playback, so it only runs one user.
-
/build buildname: The build in which to create the log folder. If the specified build does not exist, Rational TestManager creates it.
-
/logfolder logfoldername: The name of the log folder in which to create the test log. If the log folder does not exist, TestManager creates it.
-
/log logname: The name of the test log in which to record the results of running the suite. If the log exists, you must specify the /overwritelog option.
-
/overwritelog: Overwrite the test log if it exists. If you omit this option and the log exists, you will receive an error message that says the log exists and cannot be overwritten.
-
/numusers nnn: The target number of virtual testers when executing a suite that contains user groups. This is mandatory for running performance testing suites.
-
/ignoreconfiguredtestcases: Ignore configuration matching (such as test cases that specify computers with specific operating systems or other attributes) when running test cases that are in the specified suite.
-
/close: Closes Rational TestManager after running the suite. This option is useful whenyou are scheduling a suite to run at a specific time. To make the results of the suite run remain on the screen, omit the /close option.
 |
Example
The following example will run a Messaging suite R111_Messaging from a Rational project workload25 for 50 virtual users.
Listing 2. Running a Messaging suite
D:\PROGRA~1\Rational\RATION~1\rtmanager.exe R111_Messaging /runsuite
/user admin /project D:\workload25\workload25.rsp /build 0304
/logfolder Messaging /log R111_Messaging Users25#01 /overwritelog /numusers 50 /close
|
Details:
- Rational TestManager Path: D:\PROGRA~1\Rational\RATION~1\rtmanager.exe
- Suite: R111_Messaging
- Rational Project: D:\workload25\workload25.rsp
- Build: 0304
- Log Folder: Messaging
- Log Name: R111_Messaging Users25#01
- Number of Users: 50
Staggered Executions
You can use this command line execution approach to have staggered workload executions. For example, if you want to execute a reliability execution for 48 hours, here is how you could create a staggered execution.
- Create a Rational TestManager suite for a set duration, for example six hours.
- Create a batch file and use the previously mentioned command line execution syntax to call the suite execution sequentially eight times to have a 48 hour run. Note that you will need to change the log name for every execution in order to avoid overwriting the TestManager Result logs.
Listing 3. Sample batch file contents
cls
@echo off
echo
echo ***************************************************
echo ************ Test suite kickoff ************
echo ***************************************************
echo Run One
D:\PROGRA~1\Rational\RATION~1\rtmanager.exe R111_Messaging /runsuite /user admin /
project D:\workload25\workload25.rsp /build 0304 /logfolder Messaging /log
R111_MessagingUsers50#01 /overwritelog /numusers 50 /close
echo Run Two
D:\PROGRA~1\Rational\RATION~1\rtmanager.exe R111_Messaging /runsuite /user admin /
project D:\workload25\workload25.rsp /build 0304 /logfolder Messaging /
log R111_MessagingUsers50#02 /overwritelog /numusers 50 /close
echo ***************************************************
echo ************ Test suite completed************
echo ***************************************************
|
- If you want to schedule the execution at a particular time, you can schedule the batch file execution using Windows task scheduler. If you have a Windows operating system, go to Start, Program Files; Accessories; System Tools; Task Scheduler. Using task scheduler, you can schedule your suite execution at a particular time to utilize other unused machines in your lab late at night or over the weekends. To avoid receiving an IBM® Rational® License Expiration warning, open Rational License Key Administrator, go to Settings and disable Display Expiration warnings. This setting is enabled by default.
Summary
Our experience confirmed that mixing existing features with a general knowledge of technology concepts can help an automation engineer to come up with creative solutions using Rational TestManager. Configuring Rational TestManager to run test suites from a command prompt was not difficult. We recommend this approach for teams trying to run consecutive suites automatically. Teams trying to kick off longer duration executions can also consider this approach, because it allows them to resurrect failed or terminated virtual testers periodically, every time the execution is restarted.
Resources
Get products and technologies
Discuss
About the author | | | Vaibhav Telang works as a Lotus Software Engineer in the India Software Labs at IBM. |
Rate this page
| |
