Level: Introductory IBM Staff, Staff, IBM
22 Jul 2004 IBM Tivoli's Configuration Manager is an effective solution for staging and deploying code from IBM Rational ClearCase. Tivoli and ClearCase interoperate seamlessly to provide distribution capability and traceability and to effectively manage the deploment process from the staging repository to target systems.
Overview
Introduction
A typical problem customers face is the lack of deployment capability within ClearCase®. Being able to trace software assets (bi-directionally) from development, test and deployment is yet another huge problem several clients face. In order to manage and control the deployment process they must utilize other products or create their own. Tivoli® provides deployment capabilities that address this need. Rapid deployment is geared toward providing management of software assets throughout the entire development-to-test-to-production lifecycle.
Since the IBM® acquisition of Rational®, the Tivoli and Rational brands have been collaborating on how to compliment each other's solution offerings. Tivoli's configuration manager software distribution solution answers most ClearCase customer requests for assistance with deploying and staging code from Rational ClearCase.
This document illustrates the fact that the tools interoperate cleanly with each other. The document also highlights that, with the proper procedures in place, we can provide traceability between software deployed on multiple hosts to an identified baseline in Rational ClearCase. These products are two distinct entities, but can work together as one solution for customers requiring a deployment solution within a ClearCase environment.
The recommended versions for the uses of this demo are the latest versions of the solution offerings: ClearCase v2003.06.00, and Tivoli software distribution 4.0. Tivoli software distribution version 4.0 comes bundled with Tivoli configuration manager Version 4.2.1. Tivoli configuration manager is a suite of Tivoli tools to deploy and manage the inventories of staged and deployed software. Licenses and software distribution components need to be installed for the software distribution capability to work in the Tivoli configuration manager suite.
Tivoli configuration manager
Tivoli configuration manger controls software distribution and asset management inventory in a multi-platform environment. Using the software distribution component, you can install, configure and update software remotely within your network, eliminating the need to manually update software on numerous systems. You can:
- Distribute client/server applications, applications for desktops, laptops and pervasive devices across multi-platform networks
- Update existing software with newer versions
- Synchronize software on distributed systems
Using the software distribution component in Tivoli configuration manager, we can close the gap between the staging code repository and the target system. Such a feature rich solution easily does away with home grown solutions developed by Rational customers for deploying code or binaries hosted in ClearCase.
The UCM process
The UCM processes that impact the deployment activities are primarily in the build and deployment scenarios; the development scenario of UCM has minimal impact with regards to deployment.
Development scenario
In the development environment the developers will use either the standard parallel development paradigm that UCM provides (or the serial line development model). What is crucial in this environment is to understand which stream in UCM will accept changes from multiple developers for integrating changes. This is important since it allows the capability to deploy only the modified subset (i.e. for unit testing) or the whole application. This is due to the fact that, when using parallel development, multiple integration points can be setup and have sub-system baselines that can be tracked and deployed separately along with the system baseline. Using the parallel development model allows more flexibility in development than a serial model of development. In a serial model it is implicitly understood which stream assumes this role. This results in the whole baseline, in its entirety, being deployed on a regular basis. The serial model renders itself well for small projects but does not scale in an enterprise environment. Involvement from Tivoli in this space is minimal. Deployment activities are primarily focused on stable builds needing deployment.
Figure 1. UCM development Scenario
Build and deployment
This phase starts when the release team has consolidated the list of activities to baseline. Once the baseline activities are done, the build process commences. In most typical deployment scenarios, the deployed artifacts are *.class/*.jar/*.war/*.ear files or other derived objects (i.e. binaries). In some cases, deployed artifacts can be source code as well. It is assumed that all the source files are checked into the baseline. Once this build is completed and the baseline goes through its promotion levels, the derived binaries can be packaged into a Tivoli distribution package.
UCM does support the notion of promotion from a quality level so the promotion of artifacts can happen in the UCM context. In some cases, particular organizations have their own custom promotion model for code deployment. Such a process can also be accommodated. In the current scenario the assumption is that a UCM promotion model is in place before a Tivoli package can be created. Depending on the scenario the baseline could either be a composite or a single component baseline.
The binaries will ideally be deployed from a build view in which in the objects were generated and will remain as view private objects. The binary files will not be checked in to the VOB. Depending on the kind of Tivoli package created, the view can either exist or be removed.
Figure 2. Build and Deployment scenario
Closed packages
A Tivoli distribution package can be a closed or an open package. A closed package copies all the files from the source host and places the content on a central distribution serve. If closed packages are going to be used on a regular basis then staging VOBs may not be required since Tivoli software distribution manages its own staging environment that allows you to perform diff's between several stored packages. The deployment strategy is to have the software packages named after baselines in ClearCase. Having the packages named after baselines in UCM supports the traceability aspect of deployment, allowing the release team to trace the packages to identified baselines in UCM. Such a feature is essential for security conscious companies.
The packages are stored on the distribution server. Tivoli configuration manager does offer an inventory control system to manage deployment logs. Tivoli Configuration Manager manages its own versions of packages, as well. When a new package gets created with the same name it adds suffixes for version identification.
Dynamic/open packages
The other type of package that Tivoli supports is a dynamic package, which only contains pointers to data sources. For example, if a dynamic package picks up its content from a particular view looking at baseline X; once the view gets refreshed to bring in artifacts from baseline Y(i.e. rebase), then without creating a new package, the dynamic package picks the changed contents of the files it catalogues and deploys those files upon request. To make this integration work, the practitioners have to understand that Tivoli packages have hard-coded references to views. This is not a problem if we create closed packages but when using dynamic packages, getting rid of the deployment view would invalidate the packages.
Setting up Tivoli
Deployment scenario with Tivoli
A typical scenario of deployment with Tivoli is depicted in Figure 3. The only addition to a typical Tivoli deployment scenario is that ClearCase is involved. Since the ClearCase version control system has file system capabilities, the Tivoli software distribution sees the versioned artifacts as part of the standard file system.
Tivoli management infrastructure services can be used to capture files and deploy to the target environment. A collection of resources sharing one or more common sets of rules or policies is defined into a policy region. This policy region captures the network topology and defines all the hosts and their configuration details along with deployment profiles that contain the software packages. In this scenario, both the ClearCase repository and the target server are defined in the same policy region if the package is a closed package then this may not be necessary.
Figure 3. Sample tivoli deployment
Deployment workflow
In the environment described in Figure 3, the ClearCase view is looking at a read-only UCM stream containing the baseline that needs to be deployed using Tivoli software distribution. The roles involved in the above deployment scenario are the release manager, the build specialist, and the deployment specialist.
The build specialist completes the build in a UCM environment. Based on the result of the build, the build specialist proceeds to create the baseline and set a promotion level. Once the appropriate promotion level is attained, this activity is completed. It is understood that this "new" baseline is the candidate baseline for deployment to testing hosts.
The release manager then rebases his/her read-only stream to the new baseline and deploys to the testing target hosts. The files could be source files (i.e. c, c++, or java) , binaries ( *.war, *.ear, *.jar, *.class or *.o) or any other file system object. Once the release manager is satisfied that the baseline is truly a candidate for deployment, the appropriate promotion level is set for this baseline.
The release manager has access to the Tivoli software package editor on his/her workstation or source host. The release manager would proceed to create the software package for deployment with the pertinent files. Once this step is completed, the team of deployment specialists can then source the packages and deploy the artifacts to the pertinent targets using Tivoli.
Software distribution environment
Software packages
Software distribution makes use of a simplified single content packaging format called the software package. All of the techniques used to create software packages are based on this single file format. A software package contains a collection of actions to be performed on a workstation. After you create a software package and catalog it at the software distribution server, you can use change management operations to manage how the package actions are to be executed. The change management operations are inherent to the Tivoli software distribution software.
With software distribution installed, to commence with distribution activities, you add a profile type, the software package, to the Tivoli environment. You can then manage the software package within the Tivoli management framework. Software packages are imported into the Tivoli environment and are associated with profiles. Software package profiles contain references to data, and instructions about how the data is to be distributed. All references to data are resolved at distribution time.
Source host
The source host is the system upon which the software package block is stored, and the location of the resources referenced in the software package and software package definition file. Any UNIX, Linux, OS/2, Windows NT, Windows 2000, and Windows XP Professional operating system in your Tivoli environment can be a source host after Tivoli management framework and software distribution are installed. In a production environment the Tivoli framework server will be separate from the ClearCase VOB server.
The effects of distributing a software package profile are different from those of other Tivoli profiles: actions are performed on the endpoints and resources are distributed. For example, when you distribute a software package profile, you execute actions, such as adding files and directories from the source host to the target, removing files and directories from the target, checking disk space on the target, and adding Windows registry keys.
Distribution targets (endpoints)
An endpoint is a computer with the Tivoli management framework endpoint agent installed and is a possible target of a software distribution. Each endpoint is assigned to an endpoint gateway, which provides all communication services between a group of endpoints and the rest of the Tivoli environment. The gateway includes the multiplexed distribution (MDist 2) function, enabling this gateway to act as the fan-out point for distributions to many endpoints. The ClearCase view server from which the server assets are being sourced will also have the Tivoli distribution gateway component and the software package editor installed. This situation is required for both dynamic and closed packages. This is due to the fact the Tivoli region must be able to communicate with all the clients including the host from where the files are visible with Tivoli server where the region is created.
Software package editor
The software package editor is a Java-based graphical interface for creating and customizing software packages in a Tivoli managed environment. The software package are created on the ClearCase host on which the versioned files are visible to Tivoli -- in this case it will be the ClearCase build view. The release host is the system where the artifacts to be deployed will be visible to the Tivoli configuration manager using a ClearCase view. In most cases this will be the server/workstation where the release manager will manage the releases, and it will have a ClearCase view running on it.
Figure 4. Package Editor
When creating the software package on the source host, make sure that when adding files and directory to the package the path definition to the source location should be done using the view extended path name i.e.
/view/deploy_view_ucm/vob/staging_vob (UNIX)
M:\deploy_view_nt_ucm\staging_vob (Win2k/XP/2003)
After the package creation is done the package is saved as a *.sp,*.spd, *.spb file on the release host, this is the file that will be imported into the Tivoli management framework server which will be the source host for distribution (this could be the same server on which the ClearCase view resides. It could also be a separate server). The names of these packages should match the names of the UCM baselines being deployed.
The details on the package file formats are as follows:
Software package definition file (*.spd)
The software package definition file is the text version of a software package. An .spd file contains only a description of the objects contained in the package, that is, a sequential list of actions to be executed on the target system and not the actual objects or resources themselves. It consists of a sequence of stanzas, each of which describes an action to be executed. Using a text editor, you can modify an .spd file by manipulating the stanzas, then reopening the file in the software package editor and save it in another format. For example, the .spd file for Microsoft Office 2000 is very long. In the software package file format it is compressed to a much smaller size. A software package in this format is in the not-built format. Not-built format states that the actual code from the source host does not reside in the package this will only happen if the package is built inside the software package editor. Software package file (.sp)
A software package saved in this format is the zipped format of an .spd file. It contains only a description of the actions to be performed on the target system but not the files and resources necessary to execute the actions. The files and resources reside on the source host. Because the software package in this format is only a description of the software package, it is in the not-built format. Software package block file (*.spb)
A software package block bundles all the resources necessary to execute the actions contained in the software package into a standard zipped format. At distribution time, the resources do not need to be collected from the source host; they are already contained in the software package block. However, the software package block must reside on the source host. When the software package block is distributed to an endpoint, it is not stored on the endpoint, but is unzipped in the target directory. By unpacking the zipped file immediately, there is no need for additional disk space on the endpoint for the .spb file. A software package that contains all its resources is in the built format. The maximum size of a software package block is 2 GB.
Software distribution
Once the packages are created on the managed node, the Tivoli managed framework server (also known as the Tivoli managed region server/TMR) will now initiate the distribution process. The software distribution process can also be done using the command line interface for software distribution in Tivoli. As stated earlier, software distribution happens through defined regions in Tivoli. While designing the policy for a region, take into consideration the security requirements for deployment as well. The interface for the TMR is the Tivoli desktop -- shown in Figure 5.
Figure 5. Tivoli desktop
Using the Tivoli desktop, the Tivoli distribution policy region is created and, as stated earlier, all the target and distribution policies are established. In the example shown in Figure 6, the region is called BNS_demo region.
Figure 6. Policy region editor
Once inside the policy region editor, we start the profile manager application. Using the profile manager, the software distribution profile is created, and the software package is imported from the source host. Once the package is imported, it shows up as shown in Figure 7.
Figure 7. Profile Manager
Once the package is imported, the user can drag and drop the package on multiple servers listed as subscribers for the distributed software. In Figure 7, there are two kinds of packages: a closed package and an open/dynamic package. When you create a closed package, the artifacts are copied into the package. Once created, this package can be deployed without actually accessing ClearCase. The other package is an open package. This allows some flexibility from a deployment perspective. The open package does not physically copy everything into the package but has pointers in it. Hence, every time the contents of the view change (i.e. the stream is rebased) the same package can be re-used. It is dependent on a viable data source (i.e. view context must be the same). In security conscious companies this practice is not recommended, and the default approach is to create closed packages for deployment.
Summary
The integration between ClearCase and Tivoli exists because of the file-system capabilities of ClearCase. In most cases, the integration between the two tools will exist through a properly defined deployment process. This process can be automated through scripts since both Rational and Tivoli solutions are command-line rich. The automated creation of a package after a baseline has reached a particular promotion level has been done in some customer accounts.
Some of keys points to be highlighted for the deployment of this solution are as follows:
- Use the stream strategy to support the granularity of distribution being undertaken.
- Open packages can be used during the development and testing phase, but use closed packages for final distribution to production.
- While using open packages, the view server must be visible to the appropriate Tivoli components.
- Once a closed package is created it does not need access to the actual contents of the view server.
- Baselines and package names should be synchronized to support traceability.
Using these solutions along with a documented consistent process, you can provide distribution and traceability support.
Glossary
UCM -- Unified Change Management, IBM/Rational's software development paradigm implemented using ClearCase and ClearQuest
Tivoli Managed Node -- A server/workstation on which Tivoli management framework is installed
Endpoint -- The final recipient of any type of Tivoli operation
Policy -- A set of rules that are applied to managed resources
Policy region -- A group of managed resources that share one or more common policies and which model the management or organizational structure of a network computing environment.
Profile -- In a Tivoli environment, a container for application specific resources information about a particular type of resource.
Tivoli desktop -- In the Tivoli environment, the desktop that system administrators use to manage their network computing environment.
Tivoli management region -- The Tivoli server and the set of managed node gateways and endpoint (targets) that it serves. An organization can have multiple regions
References
Tivoli Software Distribution User Guide from
http://publib.boulder.ibm.com/tividd/td/ConfigurationManager4.2.1.html
Mary Lai (Software IT Architect) IBM Canada
Jennie A Brown (Jennie) IT Specialist, Rational Software, IBM USA
Malcolm D Thomas (Dudley), IT Services Specialist, Rational Software, IBM USA
About the author  | | | This article is brought to you by IBM Staff. |
Rate this page
| 
