 | Level: Introductory Sara Mitchell (sara_mitchell@uk.ibm.com), Software Engineer, IBM
John Owen (johna_owen@uk.ibm.com), Software Engineer, IBM
Katy Warr (katy_warr@uk.ibm.com), Software Engineer, IBM
04 Aug 2004 This article describes how a team of IBM developers used Extreme Programming (XP) to deliver a software component for a large middleware product as part of a larger, traditional development project. How XP impacted the success of the project, the deliverables, and the interaction between team members, colleagues, and the customer, are also candidly discussed.
| |
Get the products and tools used in this article
If you are a developerWorks subscriber, you have a single user license to use
WebSphere® Application Server,
and other DB2®, Lotus®, Rational®, Tivoli®, and WebSphere products
-- including the Eclipse-based WebSphere Studio IDE -- to develop, test, evaluate, and demonstrate your applications.
If you are not a subscriber, you can subscribe today.
|
|
Introduction
In early 2003, our team of ten developers was assigned to deliver what would be an internal component for WebSphere Application Server, one of IBM's large middleware products. We all recognized that we would need to adapt to changing requirements, as well as pick up experience and learn new technologies along the way. We decided we would need a more flexible approach to our development process to measure our progress effectively, and to enable us to break large, seemingly unmanageable chunks of work into something more understandable. An analysis of different agile delivery methods was undertaken, and Extreme Programming (XP) seemed to fit our requirements most readily.
Going against the grain of the traditional software development methodologies that made up our collective background was both risky and exciting. Then, when our management team gave us the go-ahead to try this radical, new development approach, we made a conscious decision to embrace the XP mindset wholeheartedly.
The project lasted approximately one year, at the end of which we successfully -- and on time -- delivered the component we were tasked to complete. This article describes some of the high level project elements, experiences, and thoughts that came as a result of this, our first foray into Extreme Programming.
This article assumes familiarity with the XP Core Practices, defined at XProgramming.com.
Beginning the project
The team
To begin with, our development team was made up of individuals who were keen to give XP a try. Based on books, hearsay, and experience in other projects, we set about the task of defining the XP processes to which we would adhere. Many aspects of our discussions felt uncomfortable; for example, the notion of an iterative design without considering the longer term until required to do so. However, we agreed to stick with it. We defined a basic process, according to the XP principles, and then started work. Regular reviews of this process (every week or fortnight) resulted in process improvements throughout the project.
The project
Our project requirements and scope were very vague at the outset, and so we knew that they would change considerably. XP allowed us to ensure that the complete team was involved from the start, including building our working environment and delivering initial requirements. We knew that some of these work items would be superseded, but this mechanism brought the team up to speed effectively and encouraged a strong, upbeat, and cooperative atmosphere.
The customer
At the highest level, a middleware project is defined in terms of its architecture and how its components interact. We therefore viewed the component architecture as a customer requirement. We were lucky because what we were delivering was only directly used by one other component, and so we had just a single primary customer.
The environment
We had agreed on the fundamental principles and practices regarding how we would work; for example, no product code written before it is unit tested, all code to be pair programmed, daily stand-up status meetings, adherence to coding standards, and so on. We then set about creating a suitable development environment. This environment was Eclipse-based with JUnit as the unit test harness. A continuous integration machine was set up, and then we were ready to start.
Almost.
Our final hurdle was the physical work environment; namely, obtaining an available, single room large enough to hold the team for the duration of the project. It took weeks and a great deal of management persuasion in order to acquire such a workspace, but as soon as we could finally see each other and talk to each other while working together -- in one room -- the XP process made more sense. A great team spirit developed and productivity noticeably improved. In addition, with developers and testers in the same physical proximity, a new working relationship was established, each offering their own expertise to advance the delivery of the solution.
The acquisition of the single team room was the point at which the project really began to benefit from XP.
In full swing
Project planning
Unlike more traditional development methods, where planning is performed independently by project planners and team leads, the XP team defined and owned the tasks and the associated delivery dates, making each team member feel personal responsibility for successfully meeting these targets.
We began the project with two user stories and worked, with our customer, to refine these. Next, we set about identifying the main dependencies with other teams and then looked at breaking the project down into separate releases, basing our releases on the major milestones associated with the product into which we were delivering. Initially, we had four releases, each between six and ten weeks in length, and each typically containing between 15 and 30 user stories. Later, a subsequent requirement and delivery date extension (both by the customer), meant a fifth release would be added.
At this point, a release plan was produced identifying the user stories scheduled for delivery in each iteration within each release. Based on customer input, highest priority user stories were scheduled at the beginning of the release so we could, therefore, focus our efforts on the most important functions first. This practice provided the greatest confidence that the most important functions would be delivered with success.
Delivery through iterations
We followed XP practices closely in order to deliver our code iteratively. We chose fortnightly iterations with a fifteen minute daily stand-up meeting. It took a few iterations for us to find the best day to begin an iteration and what time to make the stand-up meeting. We settled on Monday for iteration kick-off, and 1pm for our stand-up meeting. We also found that we needed a mid-iteration checkpoint meeting so we could ask the question, "How much more work have we got to do for this iteration and are we going to do it all?" Asking this question enabled us to identify potential problems early and establish appropriate remediations.
At the end of each iteration, our deliverables were packaged and delivered to the customer with supporting documentation in the form of release notes.
Tracking and monitoring status
To track and monitor our progress, we chose a low-tech, high visibility approach: we used the walls, handwritten index cards, and white boards of the XP room to record project progress and designs.
However, we still had to adhere to the standards imposed upon us by the parent project, which meant completing a number of documents that would provide status to the larger project community. Although this was extra overhead and duplication, we considered this a customer requirement; that is, our customer wanted us to track progress a certain way, and so we respectfully obliged.
Completion and delivery
The component we were building was just one part of a major product release. As the end of our project drew near, we had to begin considering what might be needed for the next product release. The decision was made to divide the team into smaller equally sized sub teams near the end of our penultimate release. As the final release progressed, we needed to transfer team members onto the next project, exactly the same as non-XP projects need to. Ever-dwindling team size made pair programming more difficult. Still, we persevered and delivered our project deliverable on time, freeing up the remaining team members to join their colleagues working on the next project release.
Impact on project administration
Practicing XP within a larger non-XP environment
Middleware products are complex ones, involving many teams and many components, all of which need to interact. Therefore, upfront planning of the high level design and architecture is essential. This contradicted the XP "just in time" design approach. However, since the high level architecture provided a better picture of how our component integrated with the rest of the product, we felt that this product architecture benefited our XP deliverable.
The team was required to adhere to all the non-XP processes used by our parent project. As mentioned, this meant some duplicate effort in documenting our progress. This also meant having to adhere to deadlines that were not compatible with the XP model. For example, although all our code was fully acceptance tested, we still had to deliver it by the parent project's DCUT (design, code, and unit test) milestone date, rather than its later FVT (functional verification test) milestone date. (Code which has been through FVT has an equivalent quality to the acceptance-tested code we delivered in each of our iterations.) It is difficult to determine how we could have influenced the project here, apart from delivering good quality code by the DCUT date and using this as a point of reference for the next deliverable.
Estimation, planning, and status
Estimation is always difficult. One benefit of XP is the refinement of estimates, beginning with high-level vague costs at the project scope phase, which are further refined during the release planning and iteration planning stages.
Getting each coding pair to provide an estimate for their work ensured buy-in by all involved, and also enabled other team members to remind the programming pair of all elements included in the cost, potentially increasing the accuracy of the estimate even further. What we did not do -- and we would likely have benefited from an XP coach here -- was accurately use project velocity to indicate how much we could deliver in future iterations. Project Velocity is a technique for measuring the team's performance, which bases the amount of effort available in the next iteration on the number of completed user stories in the previous iteration. If we had used project velocity more precisely, we would have been able to highlight estimation problems sooner, thereby improving not only the accuracy of our estimates, but our overall efficiency over time as well.
It was important to the project that status of all deliverables was tracked regularly by the Big Boss to ensure things were not forgotten. (Big Boss is an XP term to represent the person to whom status is reported; the role is similar to a delivery manager.) Coding and delivery standards helped to reduce the risk here. Additionally, ensuring that all problems, issues, and tasks (such as, writing the release notes at the end of the iteration) had an owner, meant that things would not be overlooked. The daily stand-up meetings allowed us to track progress and deal with issues, concerns, and questions in a timely manner. This compared very favourably to the more traditional periodic status meeting approach. The stand-up meetings did not require formal minutes; anything that could not be answered at the meeting were written on notes that were stuck onto the wall.
Impact on our deliverable
Productivity
In some areas, we found that our XP project might have been less productive than other iterative non-XP projects in IBM of similar size and complexity. A key factor to productivity may have been the involvement of the whole team in decision making and planning. Although this involvement was beneficial in many respects, it clearly required more resources than a non-XP project where the same job might be done by a smaller subset of team members. The breaking down of tasks into smaller, manageable subtasks might be done traditionally by an individual tackling a problem, but in our XP project this was performed by the whole team.
The voluntary nature of XP can result in a slowdown in pace and urgency. We noticed that the less glamorous tasks would take longer to complete than expected. Perhaps a more authoritative stance from the Big Boss would have helped here. Also, common ownership hindered turnaround when trying to solve complex and wide reaching problems. Splitting the challenge into smaller tasks distributed between pairs did not allow a holistic approach to problem resolution. We found that complex problems benefited from being allocated a single owner in the team, even if others in the team were involved.
It may also be worth noting that a fair percentage of our team (compared to other teams around us) was made up of relatively inexperienced developers. This factor would likely impact productivity regardless of methodologies adopted.
Toward the end of the project, we noticed a decrease in work load. We attributed this to the fact that iterative acceptance testing of the code throughout the project highlighted problems so we could fix them much earlier than we would have had we been following a traditional project plan.
Quality
The combination of test-driven development, continuous integration with fully automated regression testing, pair programming, and team design certainly improved our code quality. Additionally, refactoring was actively encouraged and allowed code to be refined and improved over time. An agreed-upon common set of coding standards ensured consistency. Lack of code ownership, inexperience, and our (correct) reliance on the unit tests, did sometimes result in inelegant code, but refactoring later in the project cycle rectified this.
Impact on the team and the customer
Educational value
Many aspects of XP are processes from which non-XP environments can benefit -- and that many experienced professionals practice intuitively. For example, our less experienced team members are now far more competent in analyzing complex problems because XP formalizes this process, and the extensive unit test suite gave people the confidence to refactor code.
Some people love working in an XP environment, others hate it. Perhaps this is due to personalities, working styles, or several other factors. Reflective thinkers can find an XP team inhibiting because it is difficult to find quiet time to sit and consider a problem; their quiet state could also be perceived as a lack of interest or capability. On the other hand, people who think out loud can thrive on an XP team, although this can result in them gaining an disproportionate level of influence.
Pair programming can be difficult due to conflicting working styles; a very methodical worker will find it frustrating to work with someone who has a less ordered approach, and so on. We also noticed that some people were consistently left with the less glamorous tasks. In most cases, this was due to an individual's tendency to contribute to the team's overall achievement, rather than choosing tasks based on personal preferences. In a more traditional environment, work might be allocated more fairly.
Those who liked XP enjoyed the open, collaborative environment, enabling team members to be involved with all aspects of the project lifecycle, including design and project management.
Impact on the customer
The most important role in all of this: the customer. In XP, the customer is the driving force. The customer needed to be coached in XP and understand what XP means to them. This was occassionally a challenge, since the customer usually specified requirements up front with no further interaction until the start of functional testing. These conflicting expectations required additional effort, primarily because we had to gather requirements using both XP and non-XP methods, and then map between the two models. Ultimately, the customer was satisfied with the result of our extreme experiment.
Conclusion
In the end, we achieved our goal: the delivery of a new component into a complex middleware product, in the required timeframe, using a new delivery methodology.
So what did we learn from this?
The XP experience was highly valuable, and dramatically changed our view of how to deliver software, particularly with regard to scenario-driven development and testing. It is easy to think that XP is just a collection of tools and techniques. In fact, it is more of a philosophy that envelops every aspect of the project. That said, XP should not be seen as a panacea for all software development. We remember well a great deal of frustration over a lack of arbitration leading to endless discussions and, occasionally, inappropriate decisions, both on the technical and project level. Also, we feel it is very important to consider the personality balance within the team since, as mentioned earlier, some people find it an uncomfortable environment in which to work.
If you want to try XP, we strongly recommend that you get an XP Coach, which is something we realize we missed out on. A coach will get the project off to a quick start and be an excellent guiding light, particularly through the essential learning process at the beginning.
Finally, it is very likely that your current project is already benefiting from practices that XP endorses. In fact, development teams in IBM already embrace iterative delivery, test driven development, refactoring, code reviews, continuous integration, automated regression testing, and common coding standards. All of these result in improved customer satisfaction through the timely delivery of high quality software. XP is not the answer to all software development projects or problems, but it does provide an alternative approach to delivery that you may want to consider.
Perhaps these insights can help you decide whether Extreme Programming is right for you and, if so, help give you an idea of what to expect before you take the plunge.
Resources
About the authors | | | Sara Mitchell works as a Web Services Development Team Lead on WebSphere Application Server. She is based at IBM's Hursley Park Development Laboratory in the UK. |
| | | John Owen is team leading the project to deliver a centralized Rational service to the IBM Hursley Park Software Development laboratory in the UK where he is based. |
| | | Katy Warr works as a Technical Lead on WebSphere Application Server at IBM's Hursley Park development site in the UK. |
Rate this page
| |
