 | Level: Introductory Joe Marasco, CEO, Ravenflow
15 Oct 2002 from The Rational Edge: Marasco's fictional software development saga, Roscoe Leroy, analyzes why iterative development is an important technique for managing any project, and one that is especially critical for software development projects.
Of all the best practices advocated by Rational, I find iterative development to be the most compelling. In this article, our old friend Roscoe Leroy takes us through his analysis of why iterative development is an important technique for managing any project, and why it is especially critical for software development projects. We learn about short vectors, applied learning, risk targeting, and how staffing profile impacts project and business success.
Going Over the Waterfall
Roscoe and I were walking across his backyard to toss a round of horseshoes. I had told him that even though I hadn't played in years, I bet I could beat him. "OK, now we'll see who's going to beat whom, Sonny" Roscoe said. "And before I teach you the right way to throw a horseshoe, I'm going to demonstrate to you why waterfall development is doomed to failure."
I had to smile. Six months ago, I was the one who wanted to talk about software development. Now it was Roscoe who kept bringing it up. Always on the prowl for new ways to look at things, I was eager to see what he had to show me.
When I picked up my pair of horseshoes, I suddenly realized that the big, omega-shaped things were heavier than I'd remembered. "C'mon, get ready," Roscoe ordered. "There's the stake. See if you can hit a ringer on your first toss."
Naturally, I felt a little pressure after my bragging. I swung the horseshoe a bit, back and forth, getting a feel for it, then took one big swing. Of course, I didn't hit the stake. I didn't come anywhere near it, in fact. My horseshoe just bounced away and landed about fifteen feet from its target.
"No fair!" I exclaimed. "You need to let me have a few practice throws."
For those of you unfamiliar with horseshoes, it is normal to take several practice tosses before you get competitive. It's like anything else, really. You need to get used to your equipment, which in my case was an arm that hadn't seen a lot of exercise lately.
"Well," responded Roscoe, "that might or might not help, because you missed for three reasons. First, as you point out, the target wasn't where your arm thought it was, due to some miscalibration
1
. Second, you will always have some imperfection in your execution; none of us is perfectly steady -- the horseshoe doesn't always go where we aim it. And third, between the time you let go and the time the horseshoe hit the ground, the stake moved!"
"Bull bleep," I responded. "Nobody moved the stake."
"True enough, but after you let that horseshoe fly, there was an instantaneous gust from a 40 M.P.H. crosswind that you hadn't taken into account. The effect was the same as if someone had moved the stake in the opposite direction," Roscoe replied. "Plus, any angular deviation gets multiplied by the distance the horseshoe has to travel, so it is no wonder you failed."
"Well, at least in project management, I don't have to score a ringer every time," I said.
"Fair enough," responded Roscoe. "That's where the expression 'Close
only counts in horseshoes and hand grenades' comes from. 'Close enough'
is often just fine in project management. What really hurts is missing
the target by a wide margin, especially after your "horseshoe" has been
in the air a long time. Yet, this happens all the time with the waterfall
approach, because you only get one toss. And typically, just as in horseshoes,
there are three critical problems at work:
- The target isn't where you perceive it to be.
- You make human mistakes in your execution.
- The target moves, to boot.
It made sense. Taking one shot at anything seemed silly. But I thought I would tease Roscoe a little by taking a completely contrarian position.
The Other Extreme
"Well, I can see your point. Suppose we just forget about all this planning stuff, and just constantly reevaluate where we are and where we need to go," I smiled sweetly.
"Stop trying to bait me, Sonny," replied Roscoe. "We know that anarchy doesn't work either -- just produces lots of senseless reaction to local events. I think you physicists call it 'Brownian Motion.' Lots of activity, little progress.
"The real issue is this: You have to do any project in a series of steps.
The crucial things you need to understand are the best length for
each step, and how many steps are required." I could see that he was on
the verge of a big exposé, and I wanted to pursue it, so I said,
"OK Roscoe, what you're getting at interests me a lot more than horseshoes,
and besides, my wrist is sore. Let's take a break."
Roscoe charitably let me off the hook, horseshoe-wise.
"Yes, let's go get a cup of coffee, and I'll introduce you to the concept of 'short vectors'," said Roscoe. "I'll draw you some pictures while we talk."
So we tramped back up to Roscoe's house, grabbed ourselves some coffee (he keeps his coffeemaker going all day), and adjourned to the back porch.
Roscoe's First Picture
No sooner had we settled into a pair of creaky old rockers, when Roscoe took a stubby pencil out of his pocket and drew this picture on a paper napkin:
"Now, we know from elementary geometry that, in a perfect world, the shortest distance between the start of the project and the finish (the 'target') is the straight dotted line I just drew. Waterfall guys are under the illusion that they can follow that path, but we have just demonstrated the three reasons why they can't. Really good project managers follow a path like the other one I drew, that goes from S1 to S2 to S3, and so on -- that involves a series of small steps before you converge on a result. I call each of these steps a vector, which comes from air traffic control lingo. You software guys would call each step an iteration."
"Looks like a sailboat tacking into the wind," I remarked.
"Sure enough, but let's just stick to the geometry for a minute. I want to contrast this path with the path a project manager might follow if he were trying to do everything in just a couple or three iterations." Roscoe took another sip of coffee and grabbed another napkin.
Roscoe's Second Picture
"Here's the path that manager would take," he said, as he drew the following:
"Note that in this case, as in the first one, we have made the simplifying assumption that the target doesn't move, although we know it always does," he added.
I noticed immediately that L1 was in the same wrong direction as S1 in the previous diagram, and I pointed that out to Roscoe. "Sure, they both get off to the same bad start because of errors in aim and execution. But notice that Mr. Long Vector, as I like to call him, stays on the deviant path much longer."
It was also true that, at the first correction point, both Mr. Short Vector and Mr. Long Vector made the same wrong "guess" for their second direction; L2 and S2 are basically in the same direction. But once again, Mr. Long Vector persisted too long before making a correction.
"Now, let's measure all three paths," Roscoe said. "The dotted line comes out to 5 inches. Let's see, hmm, L1-L2-L3 comes out to about 7.5 inches. And S1-S2-S3-S4-S5-S6-S7 comes out, let's see, to around 6 inches. So Mr. Short Vector is 20 percent longer than optimum, and Mr. Long Vector is 50 percent longer than optimum."
Wait a Minute!
I smelled a rat. "Hold on there, Roscoe," I exclaimed. "I can draw Mr. Long Vector's path so that the total length is the same, or less than, Mr. Short Vector's."
"True enough," replied Roscoe. "There is nothing, geometrically speaking,
that says Mr. Short Vector will always take a shorter total path. If Mr.
Long Vector is very lucky, he can come out better. But that is not what
we observe in practice. Statistically, averaged over many Mr. Long Vector
projects and Mr. Short Vector projects, we find that Mr. Short Vector
projects come out better. The reason for this is that short vectors allow
you to do some other things that you can't usually accomplish with long
vectors. I'll talk about those in a minute. Although you are right that
there is no geometrical 'proof' of this, it is an empirical fact, and
the geometrical analogy still has a lot to recommend it."
Just as I was beginning to accept this line of reasoning, Roscoe drove his point home: "Besides, neither of these drawings takes into account the third factor: that you are constantly aiming at a moving target. Because, unlike the horseshoe stake, targets always move in the real world. That means Mr. Long Vector faces much more risk than Mr. Short Vector, because Mr. Short Vector gets to take aim more often. As the target moves, he gets to readjust and therefore spends less time moving in the wrong direction."
Keeping the Vectors Short
"So," said Roscoe, "let's continue with our analysis. In our geometrical analogy, if time and resources are proportional to path length, we can see that Mr. Short Vector only needs 20 percent more than the minimum required for success, while Mr. Long Vector needs 50 percent more than the minimum. So Mr. Short Vector has much more margin for error.
"Another way to look at this is to measure scrap and rework. Out of the 120 units that Mr. Short Vector expends, 100 are required to get the result, and 20 are wasted, so his scrap and rework represents about 16 percent of the total effort. Mr. Long Vector wastes 50 out of 150 units expended, so his scrap and rework represents 33 percent of the total. Using scrap and rework as a metric, Mr. Long Vector is twice as bad as Mr. Short Vector." Roscoe was quite convincing.
"OK, Roscoe," I conceded. "It seems that short vectors are a good all-around principle in project management. But why is the concept even more important in software development projects?"
"I'll be glad to tell you, Sonny," said Roscoe triumphantly. "Why don't you get us some more coffee first?" Then he lit up one of his favorite stogies.
The Application to Software Development
When I returned with our refills, Roscoe took a long puff on the stogie and started in again. "It seems to me," he began, "that software development projects tend to have, in general, the following characteristics:
- Resources are very tight; there always seems to be little margin for error.
- The 'target' is hard to get in focus at the outset; just as in horseshoes, we have to get used to our swing and calibrate as we go. Another way to state this is that requirements are usually poorly understood at the beginning.
- Execution errors are unavoidable, especially at the beginning.
- The target always moves during the course of the project.
- Staying on a bad path too long is typically catastrophic; it means throwing out work you thought you'd 'accomplished' and starting over again.
When you take these characteristics into account, an iterative development approach with short vectors starts looking pretty darned important."
The argument was persuasive. It was clear that errors, particularly in early iterations, could have a really big impact unless they were detected and corrected. When I mentioned this to Roscoe, he nodded emphatically.
Applied Learning and Short Vector Direction
"You are not as dumb as you look," he rejoined. "You need to work with short vectors so that the whole team learns as it goes and makes course corrections as a consequence. When you get to calibrate your machinery as you learn, you make smaller 'aiming' errors. You get to execute better on each succeeding iteration, given what you have learned, so you deviate less and less from your intended path. And each time you combine what you've learned with the discovery that the target has moved, you are better able to anticipate future movements. You might even get smart enough to 'lead the target' a bit.
"So a short vector approach means a shorter total path for two reasons. One, you spend less time on bad paths early on (short vector principle). And, two, your later vectors are closer to optimum (learning principle.) Although there might be lucky exceptions, statistically, these are the two features responsible for shorter total path length when averaged over many projects."
Was there a systematic technique for getting maximum learning out of the early iterations? I wondered aloud.
Risk Targeting
"Yes, it's called risk targeting," replied Roscoe. "Risk targeting is about understanding where the threats are on our project. For example, suppose we are using some new technology that has never been used in production before. That has the potential to take us on a vector in a very wrong direction, because if the technology is inappropriate, we might have to spend a long time working with it, only to abandon it and replace it with another. So let's look at how this might play out in an early iteration."
Roscoe began. "Having identified 'using this new technology' as a risk, we need to rank order it with all the other perceived risks on the project. Suppose we identify it as the number one risk on the list. What do we do then?"
"As a software guy, I think I can answer that," I offered. "The first step is to remove or reduce the risk of this new technology on the next iteration. But, according to your short-vector principle, we want to limit the scope of each iteration. So we need to design a part of the system that will 'wring out' this new technology as quickly and thoroughly as possible. We want to make a decision, and we want to be clear and quick about it. Designing the iteration is a little like designing a scientific experiment: What do we want to determine? How will we determine it? And how can we do it with a minimum of interfering effects?"
"Couldn't have said it better myself," Roscoe chimed in. "So how would you answer the folks who say, 'Arggh, that's not iterative development; that's just prototyping'?"
"Sure, it sounds a bit like prototyping," I continued, "but there is an important distinction: We need to test the technology in the context of how it will eventually work in the full product, so we need to consider scalability and performance, too. 'Toy' experiments at this stage are dangerous, because they can lull us into a false sense of security and make us complacent. A well-designed iteration leads us to build a piece of the system that either validates or invalidates the technology. At the end of the iteration, we need to be able to assess our working software and answer the simple question, 'Do we continue down this path, or not?' If 'yes,' then what we built in that iteration remains a part of the product, unlike a prototype, which gets thrown away.
"And," I summed up, "the bias has to be 'What experiment will show that the new technology is inappropriate?' rather than 'What experiment will make us feel better about the new technology?' Sometimes in our desire to go down a certain path -- in this case using the new technology -- we design iterations or experiments to make us feel more comfortable. This might have the unfortunate consequence of making us feel much more uncomfortable later on, when we discover that we have been engaging in wishful thinking."
"Yeah," said Roscoe. "A good recipe is to be 'tough' early to avoid big trouble later. Using risk targeting gives us a focus for each iteration. We might target more than one risk per iteration, but we should always be addressing the risks in priority order. Remember, the earlier we start working on the hard problems, the more time we have left to solve them," he continued. "Which leads me to a huge pitfall I've seen over and over again on software development projects."
I could hardly wait.
Have You Heard This One Before?
"Sometimes I hear software development project managers say 'We'll tackle that risk later; that'll give us much more time to think about it while we're doing all the easy stuff.' Whenever I hear that, my blood pressure goes through the roof!" exclaimed Roscoe.
"Ah, Roscoe, I know where that wrong thinking comes from," I said. "It's what we tell American students about how to take tests in a time-constrained setting. Over and over again, we admonish them to do the easy stuff first, to 'get money in the bank' and reserve the remaining time at the end to work on the harder stuff. The logic is to get as much credit as you can for what you know, and 'don't leave money on the table' because you run out of time."
Roscoe reflected for a moment. "Well, that might be a legitimate strategy for certain environments. The problem is, software projects are different. You don't get partial credit for an unfinished project the way you get partial credit for an incomplete test. In project settings, you must solve the hard problems. So the situation is not only not analogous; it is diametrically opposite."
I agreed. "So what you're saying, Roscoe, is that the idea that a team will work on the 'hard' or 'high-risk' problems in the background is an exercise in wishful thinking, because:
- While the team is working on 'the easy stuff,' they will not be thinking about the deferred risks. Teams have difficulty focusing on too many things at once, and once they start working on the easy stuff they will forget about the risky stuff they deferred. 'Out of sight, out of mind.'
- Parkinson's Law says that the 'easy stuff' will expand to fill out the schedule. This will squeeze out the harder, more risky stuff. I have seen this happen over and over again. After taking 20 to 30 percent longer than expected on the soft issues, sometimes due to excessive perfectionism, the team finds itself behind schedule. Then it gets hit with the really hard problem. Which, of course, was there all the time; it didn't go away because they ignored it.
- The net result is that the team winds up having less time to
work on the hard problem rather than more time.
- And here's the ultimate disaster: Sometimes, to solve the hard problem, you have to throw away the work you did on 'the easy stuff' and do it all over again."
I concluded with: "So once again, the moral of the story is: Prioritize your risks, and attack the biggest ones first. If there is not enough time left over to do 'the easy stuff,' then you were certainly going to fail anyway."
"Now you're striking oil, Sonny," said Roscoe, and added one more cogent observation to supplement mine: "If I were behind schedule, I would feel much more comfortable going to my boss and asking for more time if I could demonstrate that my team was executing on a plan that had already squeezed out most of the risks. On the other hand, if I had to admit that not only were we behind schedule, but also that there were still high-risk items on the table, I'd be in a very weak negotiating position. I probably wouldn't get my extension, and the project might be cancelled. And maybe it should be."
More on Applied Learning
It was all coming together for me: short vectors, risk targeting, and hard problems first. One thing really hit me between the eyes: In iterative development, we are forced not only to learn as we go, but also to use what we have learned.
I pointed out to Roscoe that we can't plan the next iteration unless we take stock of what we learned in the previous one. There is no "autopilot" in iterative development; we can't be asleep at the switch. In some sense, iterative development not only encourages applied learning: It demands and requires it.
"Contrast this with organizations that don't develop iteratively," said Roscoe. "How often have you heard people say 'Well, that's a mistake I won't make on my next project,' or 'We'll have to remember that for next time'? This happens because their project plan is so rigid that they cannot incorporate lessons learned right away. They have a 'plan of record' that either can't be changed or is very difficult to change. This is tragic for two reasons: One, the team becomes resigned to 'going down with the ship' because they think it's 'too late' to change the plan; and two, because the lesson will likely be forgotten by the time the next project rolls around, and the same mistakes will probably get repeated."
The vein on the Roscoe's temple was starting to bulge, so I knew we were approaching a crescendo. "This is 'planning' run amuck," he exclaimed. It amounts to confusing the map with the territory. Or using a tool as an excuse for failure. If the current plan is going to lead to disaster, then CHANGE THE PLAN!!!" He slumped back in his chair.
I had to agree. Iterative development mandates that you immediately apply the lessons you've learned on this project to all successive iterations of the same project. As any educator can tell you, learning is most efficient and effective when you apply lessons learned as soon as possible. This is a very crucial difference between iterative and waterfall development. Iterative development insists that you "strike while the iron is hot," even if it involves pain; if you apply the right technique while the pain is still fresh, then everyone involved will remember the lesson better and longer.
Business Implications
Roscoe had one more point he wanted to make. Once again, he reached for the napkin holder and pulled out a fresh "canvas" for his final sketch.
"No doubt you have seen curves like this that contrast waterfall development with iterative development," he said, as he sketched the diagram below:
"The basic message here is that waterfall development remains high risk for much longer, because until there is meaningful system integration, we don't really know where we stand. With iterative development, which employs risk targeting and meaningful integration and build activity from the first iteration, risk is squeezed out early. In both pictures, point 'D' represents the point at which we might decide to cancel the project. That is, if the risk curve does not begin to plummet at that point in time, we might decide that we can't go on; without a diminution in project risk, maybe we cannot and should not continue to invest."
"Sure Roscoe," I said, "I have seen Dave Bernstein draw those curves for years. No news there. Clearly, it is better to make this decision early rather than late. So waterfall is the inferior approach."
"Yeah," countered Roscoe, "but there is a second, less well-understood reason that makes iterative development even more compelling. And I'm going to show it to you right now."
The Staffing Effect
Roscoe grabbed his pencil. "Let's superimpose a staffing profile on the previous curves. Waterfall projects staff up early, whereas iterative projects defer staffing, relatively speaking. Small, elite teams that can go fast are best for doing most of the work in the early phases, in other words, for Inception and Elaboration iterations. Then, you add significant numbers of people as you move into iterations for the later phases, Construction and Transition. The difference looks something like this:
"Now, let's consider a project that gets canceled at point D on each curve. The resources that will be consumed to that point are indicated by the crosshatched areas on the curves -- the area under the staffing curve from the start to point D, when cancellation occurs:
Compare the crosshatched areas. The iterative scenario is superior for two reasons: You decide to abort earlier, and you have much less invested because you started small and planned to staff up later."
Roscoe was right, but there is one small detail we should mention here. Although iterative development uses far fewer people early in the project than waterfall, in general those people are more experienced and therefore more costly. So the crosshatched areas reflect manpower loading, but not necessarily actual staffing costs. This is often a second-order effect, however, simply because of the huge differences in number of people employed.
Roscoe continued. "The implication here is that you can launch more iterative development projects, because you can cut them off early with relatively small losses. This benefit is undercut somewhat because of the limited number of really good people within most organizations who can be deployed on early iteration work. These people are always in short supply."
Roscoe was coming in for his landing. "Remember, too, that waterfall is a disaster, because as staffing costs grow -- the crosshatched area -- it becomes organizationally and politically harder and harder to cancel the project. When you cancel an iterative development project early, you have a relatively small number of people to redeploy. When you cancel a large waterfall project late in the cycle, it dislocates a large number of people."
I saw his point. In the end, if you have to cancel a project for any reason, it just makes good business sense to cancel it early, before you have too much in the way of sunk costs. And the staffing approach for iterative development keeps these costs to a minimum in the early phases.
Just Plain Horse(shoe) Sense
Often, arguments for iterative development focus on the "technology" involved in software development. We point to issues like R&D content, the imprecision of requirements, architectural risk, performance risk, integration risk, and so on, to justify developing iteratively.
What Roscoe showed me was that iterative development makes good sense from a general project management perspective. And in particular, certain characteristics peculiar to software development amplify the benefits of short vectors, risk targeting, and applied learning. So what makes good sense in general makes even better sense when the project involves software development. Furthermore, intelligent staffing during early iterations ensures that, even if you cancel it, a software development project does not need to incur significant financial losses.
As Roscoe put it, stubbing out his stogie and swinging his feet up onto the porch railing, "If it makes sense from the technology point of view and it makes sense from the business point of view, it makes sense to me."
Notes
1
It turns out that there were two very good reasons for my "miscalibration": Roscoe's horseshoes were a little heavier than the normal set, and his distance between stakes, end-to-end, was a little longer than the standard set-up. Both of these were small effects, around 10 percent, and hard to discern immediately; but they led to the target "not being where I thought it was." This is why warm-up tosses are so important: Your mind-body computer makes adjustments automatically after some practice throws.
About the author | 
| | Recently appointed CEO of Ravenflow, an Emeryville, California-based company delivering precision requirements validation for software developers, Joe Marasco served as senior vice president and business unit manager for Rational Software prior to the company's acquisition by IBM. He held numerous positions of responsibility in marketing, development, and the field sales organization, overseeing initiatives for Apex and Visual Modeler for Microsoft Visual Studio. After retiring from Rational in 2003, he published The Software Development Edge, a collection of his essays on software project management originally published in The Rational Edge. He holds a Ph.D. in physics from the University of Geneva, Switzerland, and an M.B.A. from the University of California, Irvine. |
Rate this page
| |
