 | Level: Introductory Maria Ericsson, Principal Consultant, IBM
Michel Reyrolle, Technology Manager, IBM
15 Oct 2004 from The Rational Edge: Software development capability assessments have become a tool for businesses to better understand the needs of their development organizations. This two-part article presents the reasons for scaling up an assessment from project level to organizational level, what complexities this introduces, and what added value the effort provides.
Assessments of various kinds have become a tool for businesses to better understand the needs of their development organizations. An assessment can be more or less specific; it can focus on the architecting process, or the configuration and change management environment. However, when the areas of need are broad and span many projects, most business need help in prioritizing the effort, even in knowing where to start.
This two-part article should help you understand the reasons for scaling up an assessment from project level to organizational level, what complexities this introduces, and what add value the effort provides. Our material is based on assessments we have performed with several IBM Rational colleagues for clients in the telecom, financial, IT, and medical industries. In Part I, we discuss motivation and introduce key concepts; in Part II we will present a roadmap for how to perform the assessment, based on our experiences developing solutions based on the IBM Software Development Platform.
[Editor's note: This article was originally prepared as a guide for IBM Rational Brand Services teams who conduct capability assessments for IBM customers practicing software development. While it still serves that purpose, the authors have broadened its scope to address any organization undertaking an assessment of itself, or of an external organization.]
What is a software development capability assessment?
Various types of assessments have been in the IBM Rational tech rep toolbox for quite some time. Some forms of assessment exist as off-the-shelf service products: for example, the metrics assessment package, the Rational ClearCase administration assessment package, and the software development capability assessment package.1
The concept of a software development capability assessment arose from working with clients taking an organization-wide approach to improving their development capability. The reasons can vary. Here are some common situations we have encountered:
- New technology. New technology (as in moving from a COBOL mainframe environment to Java and J2EE) forces the organization to rethink their development process and tooling environment, and the people building that organization have to learn a new way of working. An assessment helps define what practices or parts of an organization are in the most need of change and what actions to prioritize.
- Rapid growth. A software development organization has grown quickly over a relatively short time, and what used to be a sufficient, informal software development environment is no longer adequate. The organization needs to understand how to regain control of their development capabilities, and then move effectively toward further improvements. (This situation was fairly common during the dotcom era, but is rarely seen today.)
- Mergers. Merging businesses require development organizations to merge, which means sorting out different and sometimes conflicting development practices. A common development environment needs to be created, but it is typically unclear what changes to introduce first.
- Outsourcing. A development organization is outsourcing projects, or it is considering this option. Organizations usually wish to improve the way they evaluate and manage vendors. Additional challenges arise when outsource vendors are located overseas.
- Capability improvement. An organization has a general need to improve overall software development capabilities. The organization needs to understand its strengths and weaknesses, and find areas for quick returns. The organization may not necessarily wish to standardize on a common tool set, but the assessment findings may point to the benefits of that standardization.
- Market-driven process improvement. Some organizations need to improve software and/or system development processes to be competitive in new markets. Becoming certified (i.e., proving compliance with recognized quality standards such as CMM, ISO, FDA, etc.) is usually mandatory for organizations seeking a presence in certain markets.
- Systems and Products. In some industrial sectors (defense, telecom, aerospace, etc.), systems that used to be thought of simply the interworkings of mechanics and electronics are increasingly incorporating software. Collaboration amongst the various parties of a system development team is often challenged by the new technologies that also tend to increase system complexity. A way to improve is to align teams along common standards and common development infrastructures.
- Product line development. An organization may develop and maintain a line of software or system products, rather than a single one. This typically implies a high level of reuse across the product line, a process that can be repeated for each product in the product line, and development automation to help control development costs. Such repeated development lifecycles in the product line also generate demands for improvement demand and continuous optimization of the development process.
In contrast to a software development capability assessment, a project assessment is conducted to improve software development capability in the context of a specific project team, and does not necessarily consider standardization across the organization at this time or any long term gains. There are fewer stakeholders to negotiate with, less views of the problem to consider, and therefore it is typically not as difficult to agree on what issues should get resolved.
A technology-specific assessment may be organization wide, but then only focusing on the specific technology area. For example, an assessment may focus on how the organization as a whole captures metrics to determine progress and/or quality of a project.
The criteria for a broad assessment
Let's assume an organization wants to conduct a broader assessment than a project-level or a technology-specific one. What are the criteria for performing an organization-wide software development capability assessment?
Assessments of this wide scope tend to require a significant commitment by the organization as well as the assessors. For example, it is perhaps not a worthwhile exercise for organizations that are unstable for reasons other than their software development capability, since it may be difficult to retain the value of the results of the assessment.
Following are a few aspects you might consider when determining whether or not to initiate an assessment:
- Business drivers. What are the business drivers? Is it to cut cost, to achieve higher quality, or be able to deliver faster? And how are those things prioritized? Are the drivers strong enough that there is a commitment to change, and, if so, is that commitment visible at all levels of the organization?
- Stakeholders. Who are your stakeholders? Make sure that you have several views of the organization. A risk is that you spend too much time talking to "champions" -- the ones easy to get to, but who don't have the full picture of the situation.
- Organizational status. What is the general status of the organization? What is its maturity and receptivity to change? What kind of "frustrations" surface during discussions? Are those things that have to do with software development capability (where we can help), or is it mainly about other things? Is software development capability critical to the organization? Are there any previous success stories of change or improvement efforts to point to?
- Value opportunity. What does the value proposition for change look like? How big are the projects in average, how long do they typically run, and how large is the development organization? How would change in software development capability impact business results? How dependent is the organization on its capability to develop software (or systems)?
Whether or not an organization-wide assessment is the right thing to propose depends on a number of criteria. Based on the aspects listed above, here are a few suggestions:
- There should be strong business drivers and an urgency to change -- otherwise the recommendations provided may not be considered and the assessment may be perceived as not worthwhile. An organization-wide assessment should actually lead to change.
- You need to get the right people on your team involved in the assessment. Your relationships with those people need to be stable, and in turn, their relationships with the customer's organization need to be stable.
- You should devise a budget for the assessment that is relatively small compared to whatever global investments they are considering in development tools.
- There should be a vision at the organization's executive level that includes building strong software development capability.
- The organization should show commitment by creating an executive-level coalition leading the change.
What is the value of an assessment?
The primary intent of an organization-wide assessment of software development capability is to provide value to the team being assessed. An assessment performed by an external party provides an outside perspective of the strengths and weaknesses of the organization. It helps looking beyond the issues and provides prioritized set of recommendations to change based on industry best practices. The assessment process will offer key people an opportunity to formulate and discuss ideas they may not have had time to fully develop. It builds awareness of the need for change in the organization. The main point is that the development organization must provide value to its business. Better software development drives better business results. The capability assessment is a means for improving that value.
The results of the assessment (the recommendations) help motivate the investment it takes to change, and helps build the change strategy and coalition.
Ultimately, an assessment serves as an excellent basis for formulating plans for implementing a proposed solution within an organization, which fulfills the need identified during the assessment.
Stakeholders and results factors
A large number of factors influence business results, but software development capability assessments focuses only on those related to software- or system-development activities: in other words, those factors where the implementation of IBM and IBM Rational solutions can add value to the customer's business. Since the number of factors and expected results is usually large, it is critical early in the process to set the appropriate stakeholder expectations for the assessment effort. Figure 1 illustrates which stakeholders on the client side we would discuss expected business results with.
Figure 1: Stakeholders in the assessment process
As shown in Figure 1, there are a number of different stakeholders who view the outcome of an assessment from unique perspectives:
Executive management looks at results related to business vision and strategy, including IT strategy. Results should create an urgency toward change, which might affect an attitude toward business drivers, roles definition, managing change in the organization, risk management, communication, and return on investment.
Finance looks at cost drivers, contract management, cost of the value chain, rate of return, and costs of maintenance.
Supplier management consider usage of COTS, integration such as B2B, managing subcontractors, and offshore development.
Production looks at industrial process automation, control, and quality.
Sales and marketing looks at customer care (i.e., CRM -- customer relationship management), support, online services, supply chain integration, logistics, warehouse, and sales channels.
Operations is concerned with business process results and performance; integration of processes; automation and system support; process and tools support; quality management; and interrelationships across operational structure.
IT looks at strategies for components, COTS, process, tools, reuse, Enterprise Architecture Integration (EAI), outsourcing strategy, standards, technologies, legacy, and maintenance costs.
Developers want results around better software development skills, motivation, company culture, creativity, professional development, empowerment, and team cohesion.
Product management is concerned with generating revenue through product strategy, segments, positioning, packaging, pricing, technologies, architecture, quality constraints, compliance with standards, reuse strategy, quality figures, customer satisfaction, time to market, differentiators, technologies in products and operations, people involvement, and innovation management.
All these stakeholder concerns are interrelated and may reinforce each other. They offer different perspectives on an organization, and help assessment teams organize assessment questions as well as the answers. Not all of these perspectives are explored in a single software development capability assessment, but we must keep in mind the big picture of the organization while planning and scoping such an assessment.
Capability evaluation frameworks
Before presenting the assessment roadmap, we first discuss frameworks that can serve as guides during the assessment activities:
- Four collaboration areas of successful projects -- based on surveying how different engineering disciplines collaborate among themselves and with others
- Simplified software economics model -- i.e., based on COCOMO II
- Six best practices for software development -- based on surveying engineering practices
The best practices framework is generally more suitable to an organization that has, to some extent, already adopted the Rational Unified Process and Rational technology; otherwise you're probably be better off using the software economics model or the four practice areas. You may also choose a combination. Since the best practices framework is relatively more technical, you might use that with developers and project managers already familiar with Rational technology, and the software economics model and/or the four practice areas when you talk to higher-level managers.
So why no mention of CMM/CMMI among theses capability frameworks? CMM/CMMI are, in general, focused on process maturity in terms of whether process is used and improved in the organization – in other words, this is a "quality stamp" for a development organization that is used to establish credibility for their customers. The assessments we are discussing here focus more on understanding what development practices are in use, and where and how they can be improved to impact positively the development effectiveness.2
Four collaboration areas
If the assessment is done at the organization level, a framework for discussion centered on key areas of collaboration may be particularly helpful. This type of framework supports what a good development infrastructure is supposed to do: facilitate collaboration across all disciplines of a development project. Murray Cantor and Lynn Mueller of IBM Rational have done work to define such a framework, which in this paper we refer to as "four areas of collaboration": engineering, program/project management, business integration, and management of development supplier.
These collaboration areas are applicable to IT application development as well as product development. Each area can been subdivided into a set of practices, the strengths in each of which we will assess as the means for understanding the organization's maturity. The details of each of these practices may vary depending on the type of development organization under assessment.
Engineering:
Discussions about the following set of practices allow us to explore how the team collaborates in engineering the product/application:
- Use object-based system architecture and UML to capture both logical and physical design
- Use models of requirements, design, and database, not just documents
- Use a common integration repository of system engineering artifacts with product data
- Use management- and domain-specific (e.g., hardware, electrical) design tools
Program/project management:
A discussion of management practices provides insight into organization, planning, measures of success, and how results are monitored and communicated:
- Teams formed around logical and physical components, not functional units
- System architecture teams maintained throughout the project
- Lifecycle based on addressing risk, reduction of variance of cost-to-completion
- Use capability-based, use-case driven iterative development
- Use a common , organization-wide set of integrated program status, stability, quality, and financial metrics
- Earned value is tied to demonstrable results, not money spent
- Use ongoing system and component integration testing
Business integration:
Most business operations rely on computing systems, and it is vital to assess to what extent the organization thinks of and treats system development as an integral business process. Ideally, system development should reflect the characteristics and business strategies of the organization -- i.e., development capability (ability to deliver quality products on time, and on budget) should be considered critical to the success of the business. Within this area of collaboration, we assess the following practices:
- Overall business management process reflects engineering and management practices
- Use a common infrastructure across product lines to help optimize the operating environment. (Service Oriented Architectures -- SOA -- contribute significantly to optimizing business processes through distribution, integration and reutilization. As such they also help to reduce development costs.)
Management of development supplier:
In this fourth area, we assess how effectively suppliers are integrated into the lifecycle of the project, and how effectively contracts are managed:
- Supplier contracts reflect project/program management processes and development lifecycle models (not the inverse).
- Use a shared engineering collaboration infrastructure.
- Use an integrated management infrastructure.
Maturity in each of the four collaboration areas described above can be ranked on a scale from 1 to 5, defined as follows:
1 = No capabilities. Various methods and process are used with little to no consistency across development organizations.
2 = Aware. Modern development processes have been adopted by individuals who are aware of the value of these methods.
3 = Capable. Modern development processes have been adopted by various teams in some lines of business, but there is no plan to deploy a consistent process throughout the enterprise.
4 = Mature. An enterprise plan to deploy modern development processes has been developed, and has been deployed across selected lines of business.
5 = World Class. The enterprise has adopted and deployed modern development processes across the enterprise and its suppliers.
The results of the ranking can be presented as a matrix (15 x 5), but we think it is easier to visualize the results as a "radar chart," shown in Figure 2. The example in Figure 2 includes both as-is rankings as well as target to-be rankings for the organization.
(click to enlarge)
Figure 2: The more mature the capabilities of a development organization, the larger the area of coverage depicted on a radar chart such as this. The area defined by blue dots, indicating current ("as is") capabilities, shows far less coverage than the area defined by the magenta dots, which indicate the target capabilities desired.
Simplified software economics model
One way to represent the software development capability required to achieve these business objectives is to look at how product development projects perform from the point of view of the software economics involved. For the purposes of assessing capability and making recommendations, we use a COCOMO II simplified model, composed of four key software development performance parameters:
Software Development Effort = (Complexity)(Process)(Team)(Environment)
The parameters are listed in order of impact on the software development effort, and can be explained as:
- Complexity. The size of what is being built, as reflected by the volume of human-generated material, including technical material, such as source code, as well as other technical artifacts. The assessment should therefore explore opportunities to reduce the size of the software solutions, for example, by increased level of reuse.
- Process. The maturity and efficiency of the process that is followed, and in particular the ability of the process to avoid adding activities that do not add value, such as rework, communications overhead, and so on. When the Process exponent is greater than 1.0, the "diseconomies of scale"3 becomes apparent. The assessment should explore whether there are opportunities to improve maturity and efficiency of the process being executed (which is not necessarily the same as the process described for the organization).
- Team. The software development skill set, experience, and motivation of the individual team members, as well as aggregated team capability. The assessment should look for opportunities to accelerate proficiency in software technologies, best practices, and development tools.
- Environment. In the context of software development capability, this refers to the tools and techniques are used to automate the software development process. The assessment should explore opportunities to improve efficiency of the environment by integrating the tools, introducing modern tool technologies, automate best practices, and improve team collaboration.
Six best practices framework
Comparing current practice in the organization to proven software best practice is another useful way of understanding an organization's current capability. The six best practices presented below are the result of Rational's learning experiences in the software engineering field. They represent focus areas that help understand how to master the complexities of software engineering.
- Develop iteratively. To effectively mitigate risks, teams should develop software incrementally, in an iterative fashion. Each iteration results in an executable release. Architecture is validated and baselined in early iterations. The assessment should explore needs to gain control of risks and plans, for which a solution is to introduce iterative development.
- Manage requirements. Requirements will change along the way, so teams should use methods that allow them to effectively facilitate and communicate changing requirements to your stakeholders and maintain agreement with the customer. The assessment should explore things such as whether requirements are perceived to be under control, correct, of good quality, and testable. Solutions to such needs might include an introduction to use cases as a technique to facilitate requirements.
- Use component architectures. In an architecture-focused process, the goal is to produce, in early phases, an architecture that is resilient in the face of changing requirements. The assessment should explore how architecture is visible and present in the development work, how architecture is represented, and how stable it is perceived to be. Issues regarding architecture (including SOA) often have to do with how explicitly architecture is described and used, and to what extent the architecture is verified early in the development lifecycle.
- Model visually. Visual modeling raises the level of abstraction and makes it easier to communicate specification, architecture, and design. The assessment should explore any organizational communication issues, which often can be remedied by introducing more effective visual modeling techniques.
- Continuously verify quality. Software problems are 100 to 1000 times more costly to find and repair after deployment. Verifying and managing quality throughout the project lifecycle is essential to achieving the right objectives at the right time. Use techniques that make software progress and quality tangible to your stakeholders. The assessment should explore, for example, whether there's a definition of what quality means within the development organization under assessment, and whether there is a strategy to verify it; how well testing activities are integrated with the rest of the development activities; and whether testers and analysts are collaborating to ensure requirements are testable.
- Manage change. Use techniques that allow you to be in control and manage ownership, status, and consistency between the assets of the project. Controlling changes is more than just check-in and checkout of files. It includes management of change requests, workspaces, parallel development, and integration, as well as builds. The assessment should, for example, explore the extent to which the organization has common change and configuration management guidelines. For example, there should be well understood change request procedures in place, as well as good control over the assets of a project --their status as well as relationships -- and not simply the appearance of good control. Suggested solutions in this area typically include defining change request procedures, or implementing an organization-wide change and configuration management plans.
 |
Operational framework
Even though it typically isn't the purpose of a Software Development Capability assessment to suggest specific changes to the organization's structure, the assessment team needs to understand an organization's goals in order to suggest the right scope of the assessment and also to propose realistic solutions after the assessment is complete. Not all software development activities carry the same emphasis from one software development organization to the next, depending on the organizational structure and the nature of the business.
Every organization has its own flavor for how it describes its operational framework or structure, so what we present here is a generic framework for discussions and for achieving a better understanding of how and where software development related activities are being performed across the organization.
Let's consider four levels of a generic operation:
- Project level
- Product line level
- Business unit level
- Corporate level
How these levels are precisely defined must be adjusted for the specific organizations, especially since the product line level content varies greatly with the nature of the business (i.e., consider the differences in product line content among ISVs, administration teams, service teams, and organizations representing finance, defense, transportation, distribution, pharmaceuticals, telecom, etc.).
Figure 3 shows an organizational framework, with the scope of nested teams and capabilities expanding from project-level to corporate-level influence. If the assessment is conducted at the project level only, we consider it a project assessment But, in some cases where the customer is looking for a more comprehensive impact on their organization, we like to talk to several project managers, and also to people at the product line level, and even one level higher at the business unit level In some instances, this can be referred to as an organizational level assessment, although that term can be confusing since we are only looking at the software development capability aspects.
Figure 3: Organizational framework
Now, let's examine how assessments proceed at each of the levels shown in Figure 3.
Project Level
The lowest level in the framework is the project level, generally the level in the organization where individual development projects are executed. The project level is a group of activities with an assigned budget that delivers a software (or system) release. It includes not only releases of systems sold or used by the company but also releases of any system used in one or several product lines (e.g., order management, billing, provisioning, etc.). Different releases of the same product may run concurrently.
Figure 4 illustrates the types of activities and interrelationships that are commonly included in project level operations. An assessment at the project level is primarily focused on understanding and evaluating the effectiveness of software development projects against a benchmark of software best practices. This is the level where most of our assessments occur, because it is less costly, and it is easier to understand the impact of the results of the assessment. A project assessment is often very concretely tied to finding ways of helping the project successfully meet its milestones and deadlines.
Figure 4: Project processes
Product Line Level
Higher in the organizational framework is the product line level. It is formulated using the business unit level framework and extended to capture those activities and interrelationships best suited for efficient operation of the unique product line(s) of the company.
For organizations that do not develop software products, but rather develop systems to support the organization itself, this level may or may not exist depending on the complexity of the systems. If it does, it might be called the "IT strategy" level. Many of the same issues apply as you would see in a product development organization, but with different emphasis. For example, how capable you are at building systems that support certain business processes is often more important for this type of organization.
Figure 5 illustrates generic types of activities and interrelationships that are commonly included in product line operations. An assessment at the product line level is primarily focused on
1) Understanding the software development related activities being performed as part of product line management, product development, production and supply chain operations, and
2) Evaluating the effectiveness of practices being employed against a benchmark.
The number of different business processes involved at the product line level is potentially large. At this level architecture and integration of operations may be a primary focus. A strong architecture focus is needed for the competitiveness of a line of product strategy. The reuse approach impacts both the product and operations software. The assessment will emphasize the specifics of architecture strategy. The product line level may cross the concrete organizational structures such as sales, development organization, infrastructure, production, supply, and other groups involved in product line activities.
Figure 5: Product line processes
Business Unit Level
Higher still is the business unit level, which corresponds to a line of business that addresses a specific market sector. The independence and variation of operations from the corporate levels depends on the specific business and company history,maturity and size. Operations may follow the corporate policies with lots of variability; in some corporate cultures, politics plays a significant role here, which can make it difficult to get answers that consistently point in the same direction for problems and root causes. The assessment team must take care to identify the right coverage of roles and take an objective position and ensure that all views are visibly considered in the analysis of the assessment findings.
A business unit level assessment has the same goals as the corporate level: understanding business processes, business drivers and strategy, costs associated with software development activities.
Corporate Level
The highest level in the framework is the corporate level. It contains those activities defining policies, guidelines, and strategies that guide all organizations and operations of the company. Some companies have corporate policies for selected suppliers, process, tools, standards, guidelines, and standard structure. They vary among organizations according to business, maturity, history, company culture(s), and geographical distribution. There may also be a specific corporate group owning the corporate policies, for example a process and tools group or a quality group.
At the corporate level, the assessment is focused on understanding the current business strategy, business drivers and IT strategy. The organization is analyzed to understand the business processes and costs associated with software development activities.
Problem Analysis Framework
The general approach to analyze software development capability is based on identification of problems through their symptoms, research of root causes, and recommended practice.
From the assessment interviews, you should attempt to draw two key categories of information:
- The symptoms or pain points. The things that keep people up at night. Examples of typical pain points are: "We haven't been able to meet a deadline for the past two years," or "Customers are increasingly complaining about the quality of our product," or "We don't have control of our requirements, they keep changing all the time so we are not able to meet our deadlines."
- The behaviors that cause the symptoms, the root causes. The things that people and teams do (or don't do) that keep people up at night. The right combination of expertise among those who conduct the assessment interviews is essential to ensure the right information is gathered. Typical examples we hear are: "We lack a common change request process," or "We don't have a common definition of quality."
Figure 6: Symptoms traced to root causes.
Once you understand the root causes, you need to prioritize them. There are always thousands of little things than could be improved, but not all of them would have a big effect. Make sure you understand the impact of the symptoms or pain points on the organizational level you are assessing -- what they cost, and what the consequences would be if you don't do anything about it. Often, it is difficult to get concrete numbers, so you may have to look at rough criteria like how many people mention the symptom. To prioritize the root causes, you might look at how many symptoms they seem to be linked to, as well as how severe those symptoms are.
One technique to understand priorities is to look at how pain points and root causes are emphasized at the different levels in the organization, and how well they reinforce one another. You might find that a prioritized root cause at a higher level is thought of as (or made into) a pain point at a lower level in the organization. This we have referred to as the pain chain of the organization, an example of which is shown in Figure 7.
Figure 7: Example of pain chain and corresponding root causes. Root causes expressed at one level translate to root causes on the lower level. Building this chain helps understand what pains and root causes at the lower level also impact the entire organization.
To address prioritization in a more sophisticated manner, IBM researchers and consultants have jointly developed a Business Value Modeling Tool, which includes operational as well as financial metrics. This tool has been adapted for software development capability assessments, which includes COCOMO II type cost metrics as well as the five maturity levels -- from "no capabilities" to "world class" -- in the four collaboration areas mentioned previously.
The pain chain and the Business Value Modeling Tool are both tools that can be used in an assessment to help find recommendations that help improve the business value of the development organization.
Once you have prioritized the root causes, you can then formulate a solution to address those root causes. Here, it is important to point out that the framework you used to gather the assessment data is rarely the best framework for presenting findings back to the client. Part of the analysis is to understand how the client thinks of, understands, and organizes their issues. What you discover needs to be used as the client presentation framework, or you may find that the client's mode of thinking and understanding is actually part of why they struggle with the real issues. For example, the stumbling blocks in a software development organization usually have to do with communication between development disciplines. In such a case, it may be inappropriate to use the best practices framework for presenting a solution, because the real problem is not the best practices themselves, but the "interface" between the practices.
You may ask, Why is it important to go through this perhaps tedious analysis, when we already have the best practices at hand and can simply suggest them to the customer? There are two important reasons:
- Customers need to see for themselves the motivations to change, and see what the consequences of not changing would be.
- It is rarely advisable to adopt all the best practices at once. A change effort that starts by addressing only the areas of greatest need are likely to be more successful. So a complete picture, which allows all areas of capability to be examined, will in turn allow you to prioritize areas of need.
The three elements of the analysis framework should also be related to the levels of the operational framework. Symptoms, root causes, and best practices may look different at each level. You need to ensure the stakeholders at the different levels can recognize statements they have made and opinions they have expressed in the presented results. There may also be some interesting conclusions around differences in perception among those stakeholders.
Putting It All Together
We have presented multiple frameworks as tools for a software development capability assessment. During the assessment, these various frameworks can be applied as follows:
- In early discussions with the customer, determine the scope of the assessment by focusing on the operational framework of the business (project to corporate level), and where the results of an assessment would have an impact. Determine where in the organization you need to find people to interview.
- Introduce the capability evaluation frameworks (collaboration areas, software economics model, best practice areas), and determine, based on the audience for the assessment, which ones (or combination) would be most useful.
- Using the capability evaluation framework chosen and with the goals of the problem analysis framework in mind, formulate a set of questions to be answered during your interview series. Note, however, that it is not advisable to run the interview according to a strict script; you want to get the interviewee to talk freely and you may want to delve deeper into areas you didn't necessarily prepare for. But having a few planned questions help ensure that you uncover the right issues and get the information you need in order to do the analysis.
- Analyze the information you've gathered according to the analysis framework.
- To present your suggested solutions, you may organize them according to what operational level they address, or in a fashion that aligns with the client's way of organizing their issues.
Part II of this paper will present the steps of an assessment in more detail.
Conclusions
A software development capability (SDC) assessment can identify the measures that need to be taken to help a development organization provide more value to the business in which it operates. The several frameworks presented here will help in scoping an assessment along the organizational and technical dimensions. The assessment can be positioned as a component of the middleware layer in the complete IBM solution stack:
- Business: business process consulting, financial services
- Middleware: software development practices, tools, middleware, services
- Technology: hardware, systems
The level at which the assessment is performed in the operational framework is highly related to other types of assessments. At higher levels, SDC alone does not impact the organization significantly; at these higher levels, business transformation and architectural decisions within the operating environment are also key factors. In this context , the SDC assessment is one of the tools that helps in building more effectively what IBM calls an On Demand business. We envision cross-fertilization with other kinds of assessments (business process, value chain, management, architecture, and metrics) to impact more completely the value for the organization.
Part II of this paper will present a roadmap for performing a Software Development Capability Assessment, and also discuss how you would formulate an implementation plan based on your findings
Notes
1 For detailed information on these packaged services, such as data sheets and price, contact your local account team. General information is available at "http://www-306.ibm.com/software/rational/services/professional/".
2 For more background on CMM/CMMI in relation to IBM Rational technology, see the Rational pages at IBM developersWorks, "http://www-130.ibm.com/developerworks/rational/products/rup/".
3 Barry Boehm, in his book Software Engineering Economics, defines "diseconomy of scale" as a decrease in productivity based on the increasing size of a project and an increasing number of project team members (programmers).
References
Enhanced Telecom Operations MapTM (eTOM) Version 3.0. The Business Process Framework from the TeleManagement Forum Consortium (available at http://www.tmforum.com)
Walker Royce. "Improving Software Development Economics Part I: Current Trends." The Rational Edge, April 2001.
Tom Peters. Thriving on Chaos: Handbook for a Management Revolution. HarperPerennial: 1991
Barry Boehm et al. Software Cost Estimation with Cocomo II. Prentice Hall PTR: 2000
John P. Kotter. Leading Change. Harvard Business School Press: 1996
About the authors | | | Maria Ericsson is a principal consultant for IBM Rational's Strategic Services Organization (SSO). She started working in the field of software engineering and object technology in 1990 at Objectory AB, and co-authored Ivar Jacobson's book, The Object Advantage: Business Process Re-engineering with Object Technology. Since joining Rational in 1995, she has worked as a mentor and trainer in process, business modeling, and requirements management, and also spent three years as a member of the RUP development team. As part of the SSO, she currently focuses on solution deployment strategies and serves on the IBM Rational field training team. A resident of Sweden, she is based in the Kista office. |
| 
| | Michel Reyrolle is currently Technology Manager in ISV and Developer Relations for EMEA West. He started his career in research in particle physics and worked several years at CERN, Geneva. He then worked five years in Alsys on Ada compilers. After joining Rational in 1995 he has been a consultant and trainer in software development process, analysis and design and requirements management in various regions CEMA, European Strategic Services Organization (SSO) and IBM Software Services Rational (ISS-R) West. He is based in IBM Paris, La Defense. |
Rate this page
| |
