 | Level: Introductory Gottrried Luef (gottfried_luef@at.ibm.com), Senior IT Architect, IBM
Christoph Steindl (christoph_steindl@at.ibm.com), Senior IT Architect, IBM
Pal Krogdahl (pal.krogdahl@se.ibm.com), Senior IT Architect, IBM
09 Aug 2005 Explore various methodologies, such as Scrum, Extreme Programming (XP), Crystal, Dynamic Systems Development Method (DSDM), and others, that focus on the concept of Lean Software Development (LSD). In this second part of a two-part series on agile software development, the authors assess in detail their suitability to develop a Service-Oriented Architecture (SOA).
Introduction
As we illustrated in Part 1 of this series, both concepts, Service-Oriented Architecture (SOA) and agile development, aim for adaptable enterprise IT systems. However, in a CBDI notice on the topic of SOA development and agile methods, it was said that agile development and SOA, like oil and water, cannot be mixed. So, with that in mind, can you really extend agile development principles to multi-application environments like SOA? In this part of our series, we provide evidence that the answer is "yes".
SOA and Lean Software Development: A fit-gap analysis
By looking at each of the Lean Software Development (LSD) principles in turn, we investigate how you can use them to benefit the development of a SOA. In addition to this, we discuss the benefits in the context of using agile methods. The following table gives an initial high-level mapping between the principles of LSD and the fundamental benefits it brings to the development and delivery of a SOA in the context of the agile methods.
Table 1. An initial mapping of LSD and SOA principles
| LSD principle | Benefits to a SOA | | 1. Eliminate waste |
- The breadth-first service model development helps to identify right service candidates and reuse capabilities.
- Value stream mapping, a technique for systematically detecting non-valuable processing steps, lets you identify the valuable services.
| | 2. Amplify learning |
- Feedback helps to define the services for reuse.
- Synchronization is particularly useful in the implementation phase of services.
| | 3. Decide as late as possible | Leaving service interface details open leads to the possibility of learning by doing. | | 4. Deliver as fast as possible |
- Use storyboards and "SRC cards" (Service, Responsibility, Collaboration) for services to create smaller work units and higher throughput.
- Create a simple economic model of the engagement and use it to drive development decisions.
| | 5. Empower the team | By empowering and delegating micro-level design decisions to the practitioner responsible for delivering a service within a SOA, you increase the ability for the delivery of a service that satisfies the current business requirement, and not the previously predicted or stated one. | | 6. Build integrity in |
- Ensure a tighter coupling between the business processes and the delivered IT services, as well as the expected QoS (Quality of Service), by streamlining the end-to-end communication process. This way the developers can ensure that the delivered services have the required integrity.
- Ensure that the developed and deployed services are capable of delivering the expected QoS, as well as supporting a constantly changing business environment through refactoring of both the code and architecture. Controlled refactoring is possible through the services model.
| | 7. See the whole | SOA, by its very nature, is focused on business process alignment, with a strong enterprise focus. By ensuring that the engagement team maintains a focus on seeing the whole, you implement a best practice of ensuring that the service identification and orchestration is tightly aligned with the core business processes. |
Let's discuss in further detail the mapping defined in Table 1, above:
Principle 1: Eliminate waste
Breadth-first service model development
In order to understand what is important and what is waste, it's advisable to work breadth-first instead of depth-first, get an overview, and only then go into the details. When developing an enterprise-wide SOA, you should avoid building inventories of detailed analysis documents up front about insignificant parts of the system at the expense of missing the critical parts. Of course, it's difficult to foresee the future and the need for flexibility; therefore, rapid feedback is necessary.
A couple of techniques from SOMA (service-oriented modeling and architecture method) support the "breadth-first" approach well, namely: top-down domain decomposition, bottom-up existing system analysis, and goal-service modeling (see the article, "Service-oriented modeling and architecture"). After you go through the overview, decide which services to include into the Service Portfolio. Only then do you define the services in detail.
Value Stream Mapping
Mapping your value stream is a good way to start discovering the waste in processes. It focuses attention on products and their value to customers, rather than on organizations, assets, technologies, processes, and career paths. Using the results of Value Stream Mapping, you can put your energies into the processes and services that yield the most value to the organization. Pick the biggest opportunities to increase flow and value-added time. You can use this technique to analyse the business processes (to be supported by the SOA) or the software development process (for developing the SOA).
Principle 2: Amplify learning
Feedback
The "users" of services are applications. It is impossible to design a service interface for optimal reuse in advance for all the applications that will use it. Instead, you start with an initial interface and demo, then deploy the service, and finally get feedback from the using applications. A small advance team develops a simple pilot application spanning through the system by:
- Taking one business process
- Analyzing and designing the services for it
- Prototyping user interfaces
- Integrating with the backend
- Walking through the entire process
If possible, the pilot application should go into production. When the spanning application is proven in production, you know you have a workable application. At that point, multiple teams can use the same approach and drive in multiple nails at a time.
Instead of developing the entire SOA in a waterfall manner, it is even more important to rely on rapid delivery and frequent feedback, since you won't get the design right the first time. Agile approaches force rapid feedback and emergence of hot spots where flexibility is needed. Engineering practices, like test-first development and continuous integration, reduce the time for the developers to get feedback. Frequent delivery of an increment of functionality enables direct feedback from the client or user.
Synchronization
Agile approaches make inter-project dependencies visible at least and have some instruments for coping with them (see the Scrum section in Part 1 of this series). The idea of frequent synchronization between projects is particularly useful in the implementation phase of services where changes in scope of services might occur, because the services are being analyzed in detail in this phase.
Principle 3: Decide as late as possible
Leave service interface details open
In an enterprise-wide engagement such as a SOA, it seems best to design the service interfaces at a very detailed level, and let the different projects implement them. The advantage of this approach seems to be that service applications can work alone without synchronization with other projects for a long time. Unfortunately, that won't work, since you can never establish all the details up front. Doing so in the face of uncertainty results in waste (in the form of false details). Projects that have worked a long time to implement the wrong details will not be able to refactor their code easily. This results in a suboptimal, but fixed, services model.
Deciding as late as possible means leaving service interface details open until there is clear evidence of what they should look like, rather than pretending to know everything. This forces the development teams to synchronize with the rest of the company frequently, and it also results in a better services model.
Principle 4: Deliver as fast as possible
Pull systems
A pull system in agile development lets people figure out by themselves what to do by using the help of information radiators (for example, there might be a whiteboard in the hall where all the user stories in scope are written on cards; work assignment means moving a story card from the "to-do" area of the whiteboard to a "checked-out" area).
In a SOA environment, you can write user stories and services on cards. By adding services, the work units become even smaller, because as stated in the SOA Application section in Part 1 of this series, functionality is divided between user interface applications, service choreography, and a service application. The queuing theory states, smaller packages of work lead to higher throughput, such as quicker delivery of functionality.
The well-known technique of CRC cards (Class, Responsibility, Collaboration) (see A Laboratory for Teaching Object-Oriented Thinking for background) for object-oriented (OO) design could be modified to SRC cards in a SOA design. "Elements of Service-Oriented Analysis and Design" contains initial examples for such a technique.
Economic model of engagement
Traditionally, software development is seen as something that generates cost. Recently, software development is seen as something that generates revenue that can help to exploit economic options. Options-based software economics draws analogies from the financial markets: short iterations that deliver running software are seen as real options. Just like financial options, real options provide the benefit that you can buy the chance to gain from an uncertain future by investing in just a little bit now.
But it is not necessary to go that far: It will even benefit a SOA engagement if you create a simple economic model of the engagement and use it to drive development decisions. With the economic model in their hands, the team members are empowered to figure out for themselves what is important for the business: They can all work from the same assumptions. If you consider eliminating features, your marketing department might speculate that they would sell "X" percent fewer units without these features.
Principle 5: Empower the team
As with any software development, the person most educated to make a decision is the one at the front line who is doing the work. While at the solution architecture level there is still a need for tight control, the development of the underlying service should be as flexible as possible. By empowering the practitioner closest to the implementation to make micro-level decisions, you can ensure that the resulting code for a service satisfies the required functionality. Within an enterprise SOA, you can benefit from empowering and delegating design decisions to the developer of a service (within the defined interface).
Whenever you delegate decision-making power down the hierarchy, you have to make sure that all decision makers understand the overarching vision. Decision making should always be more a participatory activity than a lonely one. The environment of a cross-functional team (present in agile projects) facilitates collaborative decision making.
Principle 6: Build integrity in
The way to build a SOA with high conceptual integrity is to have excellent information flows from the customer to the development team, and between the upstream and downstream processes of the development team. From our perspective, we see value in this principle at both the service development level as well as at the service orchestration. By maintaining a good level of communication between the business and IT practitioners, you are ensured that the delivered IT solution satisfies the current and future business processes and requirements. With the tighter alignment between business and IT within a well-structured SOA, there is an increased need for the delivered services to be able to both satisfy and closely align with the over-reaching business processes. Without this high level of integrity, there is an increased risk that the delivered services will not satisfy the required business needs or QoS.
Refactoring
The practice of refactoring developed code has been recognized within the IT industry for some time now. Within the context of a SOA, this practice is just as important. With the close alignment to the business and the need to ensure that the delivered services can adhere to an ever changing and agile business environment, there is an inherent need to ensure that the design of the underlying services are reviewed and refactored on a continuing basis. Failure to do so could result in a rigid and inflexible service, which does not lend itself to supporting a changing business environment. As stated in the Agile architecture section in Part 1 of this series, the service model is an excellent tool to control continuous refactoring.
Principle 7: See the whole
One of the fundamental principles of a SOA is business alignment at the enterprise level. As with any Enterprise Architecture engagement, there is an inherent need to ensure that you always maintain both an overarching "city planning" view as well as focusing on the detailed design of the individual services to be deployed within the SOA. By falling into the temptation to optimize individual parts (or services) at the expense of the whole, you run the risk of delivering an inflexible SOA, which is not aligned to the key business processes within the enterprise.
Conclusions and outlook
As we have shown in our series, a SOA can greatly benefit from the well-documented principles of agile software practices and LSD. This match of two practices, agility and SOA, is based on a commonality -- both especially try to deal with change. A service interface should be regarded as a requirement to the implementing application, which is likely to change. Agile projects are well prepared for changes in the service interface they are required to implement.
"Continuous" refactoring of a SOA would mean constantly modifying service interfaces and service composition. This kind of agility in a SOA would require agility in the implementation projects. Based on the points highlighted in this article, we would state that true agility in a SOA works when the implementation projects adopt an agile approach and embrace change.
Oil and water normally do not mix. Admittedly, there are cultural differences between the idea of extreme programming and the controlled establishment of an enterprise-wide service architecture. But this was only the first assessment, and after having written our article, we can state that there is an emulsifier, namely the principles of LSD.
So, to conclude, we would state that in the context of SOA and the LSD principles, "oil and water do mix"!
Acknowledgements
The authors would like to acknowledge the valued input and feedback of Olaf Zimmermann, Rick Robinson, Keith Jones and Norbert Bieberstein.
Resources
About the authors | | | Gottfried Luef is currently working as an IT Architect and Consultant at IBM in Vienna, Austria. He supports various projects in the government area with expertise in Web services, SOA, and J2EE architecture. He participated in the development of Austrian government standards for Web services. Gottfried holds a doctoral degree in Information Management from the University of Vienna, Austria. You can contact him at gottfried_luef@at.ibm.com. |
| | | Christoph Steindl is a Senior IT Architect and Method Exponent at IBM in Vienna, Austria. He has been working for IBM since 2000 on a variety of software development and systems integration projects. His areas of expertise are in application development, software engineering, and methodologies. He is very knowledgeable about various agile methodologies and has given presentations about topics like LSD, agile project management with Scrum, supporting XP with Scrum, distributed agile, test-driven development, practical use-case modeling, and estimation in agile projects. He is an appointed lecturer at the Johnannes Kepler University in Linz (Austria) and at the University of Applied Sciences in Hagenberg (Austria), as well as a Certified ScrumMaster. He holds degrees for Computer Science and Mechatronics and a doctoral degree in technical sciences from the Johannes Kepler University in Linz. You can contact him at christoph_steindl@at.ibm.com. |
| | | Pål Krogdahl is a Senior IT Architect and Method Exponent with the IBM EMEA WebSphere Lab Service team. He has been working for IBM since 1998 in various areas such as software development, technical pre-sales consulting, and solution architecture. His areas of expertise are in distributed computing, middleware, and Application Services Architecture, with focus on Enterprise Application Integration (EAI) and SOA. Pål has recently written extensively on the subject of SOA-based EAs, and their relationship to IBM patterns for e-business and the On Demand Operating Environment. To date, he has completed several residencies with the IBM International Technical Support Organization, where he has co-authored IBM WebSphere-related RedBooks with a focus on Web services and SOAs such as Patterns: Service-Oriented Architecture and Web Services (ISBN 073845317X). A large majority of his project experience has been predominantly within a Rapid Application Development environment. You can reach him at pal.krogdahl@se.ibm.com. |
Rate this page
| |
