 | Level: Introductory Editorial staff (dwinfo@us.ibm.com), developerWorks, IBM
18 Apr 2006 Nobody likes the topic of IT governance, but try living without it, especially in today's service-oriented world. IBM technical leaders share with you why you must care, or live in a world of IT chaos.
Introduction: If you don't like the government, make your own
We all like to complain about the government. But now we can actually do something about it.
No -- I'm not talking about some new political movement or a revolution. I'm referring to IBM's recent push into the area of IT and SOA governance. In case you missed it, IBM recently introduced solutions to help you establish governance over your IT systems. Governance provides a framework of appropriate rules and process -- particularly when it comes to Service-Oriented Architecture (SOA) -- so you can develop quality software that meets the needs of your business over the long term.
You may be thinking: "I've had it up to here with process -- who cares about governance? I've got my hands full already." Admittedly, when we first came across this topic, it didn't exactly cause our minds to race with curiosity.
So, to help you understand why IBM cares about this (and to enlighten ourselves), we thought to involve our panel of expert IBM IT architect types. We asked them the following question:
What is IT governance, and why should the architect care about it?
Leave it to our experts -- their answers will interest you, entertain you, and possibly move you to action. If nothing else, you will come away with an understanding of an important new arena for software engineers and useful ideas for how you can improve your software development, and business, processes.
We invite your comments. Tell us what you like and don't like, what you found useful or what was useless. And always let us know what questions you have for our panel of experts. Contribute to our IT discussion forum, or just drop me a note.
Paul Dreyfus
developerWorks SOA and Web services
pdreyfus@us.ibm.com
In a world without governance...
Let's say you develop a nifty little service to convert a monetary amount from one currency to another. It starts out as a set of reusable functions within an application, then becomes a JAR reused in multiple applications, and you finally make it a separate little program that's always running and can be invoked through a remote API. That's a service.
Now that your service is available, not only do you use it, but some of your coworkers start calling it from their programs. Before long, unbeknownst to you, programs you've never heard of in other departments of your company are using it. In fact, so many programs are using it that your department has to buy and administer more hardware to run it better. One weekend, the machine crashes, and you find out about it because someone at your company you've never heard of calls you at home and makes you come into work to get the converter running again. How'd you become responsible for all this?
Now let's consider the flip side. You're developing a program that needs to convert currencies; a coworker tells you about a service your program can call which does this. Great! You get your Web site into production in record time. Then one weekend your Web site stops working -- it can't display prices in other currencies. You have to come in on Sunday to fix your program, just to discover the problem is that the currency converter service isn't running. The problem isn't with your program, but you're getting blamed because your program isn't working. How did this become your fault?
Welcome to the world of service governance. Or in this case, the lack thereof. Service governance is what helps you, your services, and those who depend on them not get into situations like these. As you can see, if you want to avoid problems like this, service governance is important.
Briefly, let me make a few more specific points, which will help you think about your own services governance scheme:
- Governance manages what the providers are committed to and what the consumers can rely on. In other words, governance helps providers and consumers set and live up to expectations, taking the element of surprise out of the system.
- Governance is more of a political problem than a technological one, like a business contract between the providers and consumers. Both sides of the agreement relinquish a small amount of control in the short term in order to gain the control they need over the long term.
- As soon as the politics are worked out, the technology that is used to invoke the services can also be used to enforce the governance. Technology can limit which consumers can invoke a service and when; it can warn a consumer that the service has been deprecated; and it can measure the service's response time; and so on.
- One good way for the technology to enforce governance is through the use of an enterprise service bus (ESB). Expose your service in such a way that it can only be recognized and invoked through an ESB, and let the standardization that comes with the ESB take care of the governance.
| | Service governance is what helps you, your services, and those who depend on them not get into situations like these. |
|
All that said, a final word of warning: Governance may become a, if not the, whipping boy of SOA. Like performance, governance may become an overwhelming concern and an excuse for every problem and justification for every questionable solution. It'll be a rhetorical hand grenade you can toss into any SOA discussion; then watch the discussion grind to a halt. A challenge for SOA is to use governance judiciously to make SOA work better, without letting concerns about governance overwhelm everything else.
Paving the way for SOA adoption
IT governance is focused on the alignment of the business strategy to the services provided by the IT department. IT governance ensures the linkage between the business process and the IT processes by establishing the management mechanisms necessary to ensure predicable IT service delivery. IT governance has traditionally been the domain of the CIO, CEO, and a handful of IT and line of business (LOB) executives.
SOA governance is an extension of IT governance. It differs from IT governance in that it is specifically focuses on the effective implementation of IT delivered through services or services-oriented computing.
The goal of services-oriented computing is to deliver the business processes as IT services. Unlike IT governance, SOA governance requires greater coupling to the lines of business. Service orientation requires a hierarchical approach to implementing governance. SOA governance must be implemented at the enterprise level and aligned with the traditional IT governance model. But unlike traditional IT governance, SOA governance requires both a layered and cross-organizational model/domain model. Therefore, the LOBs, significant programs, and communities of interest also require their own governance models. This is necessary to facilitate the realization of services that are valuable to a particular community. Likewise, SOA governance within the technical IT domain requires unique focus in order to effectively develop, deploy, and manage services.
| | An SOA strategy requires business process experts, seasoned IT architects, and a strong SOA governance program. |
|
IT governance best practices and processes have been documented in such work as CoBIT (Control Objectives for Information and Related Technologies). This work was initiated and partially developed by financial analysts. While a significant body of knowledge and best practices exist for IT governance, this is not so for SOA governance. SOA governance is an evolving science that will require a wide range of contributions from experts across industries, computer scientists, and vendors. In fact, vendors like IBM are currently developing tooling to facilitate the deployment of SOA governance. Business and IT experts will be required to work closely together in order to deploy SOA successfully. Delivering business value through service orientation will effectively change how an organization interacts with the IT department. Organizations looking to adopt an SOA strategy will require business process experts, seasoned IT architects, and a strong SOA governance program. Effective governance will be the most effective method to facilitate the adoption of SOA across the IT industry.
 |
Steady as she goes
IT governance, like governance in any other field, is all about keeping the ship steady, on course, and steered in accordance with the predefined guidelines and policies. Some projects lack clarity of direction. More fail for lack of the ability to trace engineering work to requirements, lack of following processes throughout the life cycle, and lack of documenting decisions. Every architect should care about governance on any project, more so in large complex SOA projects with multiple integration points.
Technologies change quickly, and resulting changes to the system can cause the foundations to be built strongly, if proper governance is established. Governance really means a good set of processes, and a good way to enforce them. It also means the ability to evolve the processes and policies as needs change. It is absolutely imperative for the architect to care about governance to keep things in order.
A context for entrepeneurship, achievement, and execution
A few random thoughts about why architects should care about IT governance:
History, and the world today, is full of lousy governance approaches. Precious few have prospered. So developers and architects should, and will, have innate skepticism when industry titans like IBM starting throwing their weight around on the topic of IT governance. Companies in the software business are also full of lousy governance approaches. Internal to IBM, we probably look just a little better than the industry average when it comes to our own quality of governance.
The United States governance model succeeded because it was derived from a primary perspective of promoting freedom of its people. In my view, freedom and empowerment of architects/developers needs to be the primary goal of IT governance as well, or it will fail. I relate architects to the entrepreneurs in our society.
Governance works in the U.S. (and elsewhere) because the people generally have a stake in the outcome. That is, they own property and businesses where they live and where they want their families to live.
| | Good ideas from architects cannot come to fruition unless there is a good governance model that provides a context for entrepreneurship, quality achievement, and efficient execution. |
|
Governance has many levels: Certain things can be macro managed, like national defense and social security retirement programs. Others can be micro managed, like the Neighborhood Watch program. And there are several levels in between, including education, policing, infrastructure, and economics. IT governance will be similar. We need macro management (portfolio management, business strategies, skills investments), micro management (open source communities, technical communities, configuration management, process rightsizing, artifact construction), and other mid-level governance approaches (project management, scope management, resourcing, asset management, building codes, education, etc.).
Good ideas from architects cannot come to fruition unless there is a good governance model that provides a context for entrepreneurship, quality achievement, and efficient execution. We need more good examples of good governance (and matching business results) to show others the way. Would you put up the current governance model that you live in today as an example of an industry best practice? If so, let our editors know or post your thoughts to the developerWorks IT Architecture discussion forum .
Keeping the business of SOA on track
There are at least two aspects of SOA governance. SOA governance means simply to govern an SOA initiative; for instance, communicating corporate policies to developers implementing services and giving them the tools they need to adhere to those policies. But there is a broader, more strategic aspect to SOA governance: SOA governance in the context of a Service Oriented Architecture that has been implemented as an enterprise architecture. This means that SOA is not just a single application initiative, but one that spans the entire scope of business and IT.
SOA governance is business oriented. When a company moves to SOA, they want to ensure continuity of existing business operations, manage their business model, increase efficiencies, and reduce operational cost. The impact of ungoverned SOA projects can be significant to a company's operational efficiencies and bottom line. The failure to govern SOA initiatives can result in millions of dollars in costly service redesigns, maintenance, and project delays. In an era where proving the value proposition for SOA to senior management is sometimes difficult, it can take only one failed project to provide enough doubt and skepticism to derail a company's SOA objectives.
Managing change to services
What makes governance interesting is a good question. As Bobby Woolf said, "Governance is more of a political problem than a technological one," and as Kurt Bittner said, "The most successful open source communities have strong governance models." So what is new and cool here? Perhaps it is the new techniques that are emerging in the design of composite applications. A "meet-in-the-middle" approach works really well to ensure fewer disconnects. Do a top-down business process analysis to identify the business processes at the highest levels, and break them down into more granular services. At the same time, take a bottom-up look at the applications and infrastructure that already exist in the organization to best enable reuse.
Applications are no longer monolithic: they may call services provided by a third party. In that case, how do you manage changes to those services? Do you have to change your composite applications and redeploy? While it is necessary to establish a governance framework with clear decision rights, it is equally important to examine new best practices and middleware that can support these dimensions. The article
"Versioning and dynamicity with WebSphere Process Server" introduces you to new concepts such as selectors, business rules, and process versioning that can help with some of these issues.
Without steering, there would be chaos
Having gone to Catholic school, I look words up. In my dictionary, governance is defined as "the action of governing." If I had defined a word using a form of itself, one of the nuns would have done something unpleasant to me. So, I looked up govern. One definition is "Verb: Require to be in a certain grammatical case, voice, or mood." Now that would be fun in an IT department. One cool thing about dictionaries is finding out the root of the word. The root for govern is the Greek word for "to steer."
Applications, business processes, data models, business policies, and infrastructure change and evolve. Without steering, there would be chaos. It is somewhat surprising that our industry is only recently focusing on governance. Governance is intrinsic to all other engineering disciplines. Architects and civil engineers very carefully govern changes and extensions to buildings, for example. In many cases, government inspection and legislation apply. Ungoverned changes to buildings, automobile designs, electronics, and so forth, can get people killed. In some cases, ungoverned changes to software or data can also harm people. In such environments, like embedded avionics systems, there is a long history of governance.
In less life-threatening environments, governance steers and controls the changes and use of business policies, data, processes, and applications. Governance applies the methodology and model we articulate for business processes to the processes of IT. In the same way that there are business processes, data, and policies for managing purchase orders, customer complaints, and the like, there are business processes for managing application and infrastructure evolution.
Why should an architect care? This seems like a silly question. Why does a building architect care about the governance of building change and extension? Why does a citizen care about the nation's governance? The building architect and citizen participate in the process. The process is important, and does not function without them.
Emphasizing intentionality
I would like to chime in on one point. I especially like the notion of intentionality introduced by Grady. Depending on the culture, some activities benefit from tighter control than others. For example, in some of our clients, a looser, community governance approach may make sense for development while a tighter approach may be needed for deployment. Governing well includes tailoring the governance approach to the situation.
Our challenge is to develop assets to enable our clients with services and infrastructure to govern well and efficiently and then to realize their governance approach is efficient organization practices. We have much to do.
Learning from the "gotchas" and the "ahas"
At first blush, the word governance makes most IT practitioners groan and agonize: you start imagining piles of unnecessary paperwork, cabals of micromanagers, spindles of frustrating red tape, and wheels of wasted time.
But if you revisit the portfolio of your past projects, you'll remember the heartburn of the unanticipated missteps, the flailing for nuggets of guidance/best practices, the deafening chaos at project milestones, the inconsistent expectations of the various project stakeholders, and so on. Wouldn't it be great if we could distill these gotcha! experiences and also add in the aha! moments and reference the result for future projects? Extrapolate this result, aggregate and synthesize it across all the IT practitioners in your project, in your department, in your company, in your industry vertical, and so on. Now, this starts looking like a wonderful mechanism to consolidate experiences, cross-pollinate ideas, enable well-tested decisions, and steer you away from troubled waters to deliver project solutions in an optimized fashion. This is what IT governance is all about.
Since SOA encompasses both business and IT, SOA governance extends this mechanism to include business models and corporate operations, helping build an ecosystem where people, business, and IT are fully aligned.
Obviously, instituting a well-functioning SOA/IT governance infrastructure is not a trivial task. It requires a delicate structure that needs nurturing from senior executives and requiring policies with some level of checks and balances and regular updates to the governance model and elements.
Predictability -- with creativity
At its best, governance encompasses a set of minimal mechanisms that ensure that the right decisions are made by the right stakeholders at the right time. Governance should do the following:
- Encompass only essential decisions. There must be degrees of freedom offered for the resolution of all other decisions.
- Enable decisions to be made by a minimal set of stakeholders, but created with and properly communicated to all other stakeholders in a timely fashion.
- Assure decisions are made at the right moment. Binding decisions too early will restrict innovation; binding decisions too late will induce scrap and rework.
You have to find the proper degree of governance commensurate with the subject domain, development culture, and risk profile. In one context (for example, when human lives are involved), tight governance is essential. In other contexts (such as in skunkworks projects), tight governance will strangle the organization, leaving it gasping for innovation. In any case, the best form of governance leaves it part of the atmosphere of the development community, made manifest in organizational structures, clear lines of control, appropriate governing bodies, and supported by tools that do not intrude upon a developer's day. Governance is about intentionality and accountability and serves to promote predictability and repeatability, while still allowing creativity to flourish.
The best governance governs least
There is a trend toward, as well as increasing demand for, increased agility and responsiveness in software development projects. Business stakeholders increasingly insist on measurable results in exchange for funding, while at the same time developers increasingly insist on greater autonomy and freedom to be creative. Both of these forces lead to an increased need for accountability, both from the developers to the business, but also from developer to developer. Governance is the mechanism by which accountability is established and enforced.
Lest this sound bureaucratic, it is instructive to recognize that the most successful open source communities have strong governance models that determine who can submit changes and how submissions are assessed for worthiness for inclusion in official builds. The communities vary in the formality of the governance models, which range from explicit to implicit, but governance is the mechanism by which the community regulates and polices itself. Without governance, projects tend to degenerate into chaos.
But all governance models are not created equal. While no governance results in chaos, ineffective governance introduces excess work and inefficiencies that distract the project team from the important work at hand. As Thomas Jefferson noted, the best government is that which governs least. This is not an argument for no governance, but rather an argument for governing the things that matter and leaving the rest up to the judgment of the individual. Bad governance models distrust people, leading project team members to respond in kind. Good governance models rely on holding people accountable for their commitments while providing the flexibility to be creative in meeting those commitments.
| | Governance is at the heart of how a team works. |
|
The right discussion to have is not whether a project should have a governance model, but what sort of governance model the project should have. Should anybody be able to make changes to code? Should anyone be able to add a new feature? If there are restrictions, who will enforce them? And how will decisions be made? Good governance practices are often similar to those followed by successful open source projects: The community itself (or at least its leaders) decides how it will govern itself. Governance models handed down to the team are often self-defeating if the team does not buy into the model. The team needs to see a clear tie between the governance model and improved results. In other words, the governance model should measure things that matter and hold team members accountable for things that will affect the overall team results.
If the governance model is something that is imposed from the outside -- if team members only follow it because there is some "governance cop" watching them to make sure that they are "following the rules" -- there is something wrong with either the governance model or the team. Most likely there is something wrong with both. A team that does not recognize the importance of having a way to organize and make decisions is probably too immature to successfully deliver significant results. A governance model that has no value to the project team is probably too weighty and bureaucratic to create the conditions for success, and it may even increase the chances of failure. The best models will reduce complexity by making the decision process clearer, and by doing so, improve the project's chances of succeeding.
Governance, discussed in these terms, may provoke the reaction, "That's nothing new; the best teams have always done those things." Giving it a name provides a way of talking about something that is essential to any group endeavor. Governance is really at the heart of how a team works; it is not something imposed from the outside, but it is something that every team needs. Discussing and agreeing on the governance model can be an important team-forming and norming activity, and can help to create better ways for the team to work if the governance model comes from within the team.
So that's why governance should be important to software developers.
The umbrella and the flashlight
Here is my understanding of this very important topic, starting with some semantics behind my ideas:
Governance. The people, processes, and infrastructure in place to ensure compliance with a set of policies to ensure repeatable results in execution, service enablement/run time, and product creation.
SOA governance. The governance of the service life cycle to ensure that the needs of the business are supported by a set of flexibly recomposable IT services or components. SOA governance relies on a process of oversight composed of policies, principles, checkpoints, and reviews, the execution of which is delegated by management.
It's important to differentiate between various aspects of SOA governance:
- Run-time governance: How can I monitor and manage service execution and compliance at run time?
- Process governance: How do I insert control points and compliance criteria, similar to adding Capability Maturity Model Integration (CMMI) control points?
- Organizational governance: How do I structure my projects, my teams, my IT, and my business? How do they communicate and cooperate to achieve business-defined objectives in a measurable manner?
- Financial governance: How do I fund service projects? Who pays for them? Domain owners? If so, how do I decide who the domain owners will be (from the organizational governance body just mentioned) and who will spend money on a given service required by one domain but not yet offered by another? (See the Virtual Provider pattern for one solution approach.)
Clearly there are overlaps among the aspects; however, they exhibit sufficiently unique characteristics to be differentiated from a practical point of view.
So I see SOA governance -- governing with and through SOA -- as an umbrella over the service life cycle. It connects the various facets that are out of scope to methods and picks up the ball that is delivered by methods after execution. Service realization includes implementation, testing, provisioning, deployment, monitoring, management, and governance. The latter linkage allows governance to pick up where methods would otherwise leave off.
SOA governance is a welcome umbrella of protection from too narrowly focusing issues (such as service refactoring after service realization and using variation-oriented analysis and design) after methods have delivered their work. Not only is it an umbrella, it provides a beam of insight into aspects that are beyond the scope of individual projects and looks across the enterprise. For example, it can help you get a broader view of the set of reusable services that can be used across the enterprise and help you designate domains for ownership of the services.
About the experts
Bobby Woolf
Bobby Woolf is a member of IBM Software Services for WebSphere, consultants who help customers achieve success with WebSphere products. He is a coauthor of Enterprise Integration Patterns and The Design Patterns Smalltalk Companion. Read more at Bobby's blog on developerWorks.
Andras Robert Szakal
Andras Robert Szakal is chief architect for the IBM Federal Software Group and is an IBM distinguished engineer and senior certified IT architect. He is also serves on The Open Group Board of Directors.
Sridhar Sudarsan
Sridhar Sudarsan is a senior IT architect with Software Services for WebSphere. He has led enterprise architecture solutions for several customers worldwide, including large companies in the finance, public sector, automobile, and SRM industry verticals. He is a cocreator of the batch programming model in J2EE that is now a component in WebSphere XD, and now he's evangelizing it with customers and building a practice around the technology. He is currently leading a large-scale SOA Center of Excellence at a large insurance company.
Walker Royce
A long-time member of the Rational Software team, Walker Royce is currently vice president of IBM Software Services - Rational. He is the author of Software Project Management, A Unified Framework (Addison Wesley Longman, 1998) and a principal contributor to the management philosophy inherent in the Rational Unified Process. Before joining Rational, Walker spent 16 years in software development at TRW Electronics and Defense, where he received the Chairman's Award for Innovation.
Calvin Lawrence
Calvin Lawrence is an executive architect on the IBM Software Group Emerging Technology team. His responsibilities include advancing strategic IBM architectures, technologies and products in support of key strategic initiatives and ensuring the success of customer implementations using IBM technologies. He is former chair of IBM Software Group Worldwide Technical Leadership Council.
Christina Lau
Christina Lau is an architect on the On Demand Development team. Her current projects include creating Pattern Solutions using Rational Software Architect and piloting business innovation and optimization functions. Christina is a senior technical staff member and a member of the IBM Academy of Technology. She is also a coauthor of a new book: Introduction to IBM Rational Application Developer.
Donald F. Ferguson
Donald Ferguson is one of 53 IBM Fellows (IBM's highest technical position) in IBM's 200,000 employee technical team. Don is also the chief architect for IBM Software Group (SWG). Don chairs the SWG Architecture Board, which oversees the architecture and integration of WebSphere®, DB2®, Lotus®, Tivoli®, and Rational® products. Don was the original chief architect for the WebSphere family of products. He joined IBM Research in 1985. His interests include raising and playing with his children, distributed systems, simplifying application development, systems management, Web services, transaction processing, performance, and karate. Read his blog, Middleware and tools.
Murray Cantor
IBM Distinguished Engineer Murray Cantor promotes and extends Rational best practices, and works closely with customers on innovative ways to build and deliver systems more efficiently. Currently, he leads the evolution of a new engagement model for transforming software development organizations, as well as Rational Unified Process for Systems Engineering® (RUP-SE®). He also focuses on how to integrate IBM Rational field capabilities with those of other IBM brands. Murray has published two books and numerous papers, and plays a key role on standards committees relating to UML and RUP. He received his Ph.D. in mathematics from the University of California at Berkeley in 1973.
Sanjay Bose
Sanjay Bose works for the IBM Software Strategy division and leads the Enterprise Integration Design Center to identify IBM software portfolio requirements, and develops solution components and assets by engaging enterprise clients and IBM software product development laboratories. He has more than 12 years of IT industry experience, primarily focused on creating product architecture and design, articulating technical strategy, and designing enterprise application systems using distributed technologies. His areas of expertise include SOA, Enterprise Service Bus (ESB), Web services, Java™ 2 Platform, Enterprise Edition (J2EE), and e-business technologies. He has coauthored the book, SOA Compass, and has published articles on IBM developerWorks and Systems Journal. He lives and works in Pittsburgh, Pennsylvania, and his spare time is spent with philosophical ramblings, books, movies and his Sony PlayStation. Read his blog, SOA, ESB, and beyond.
Grady Booch
Grady is an IBM Fellow who has served as architect and architectural mentor for numerous complex software-intensive systems around the world in just about every domain imaginable. Grady is the author of six best-selling books and has published several hundred articles on software engineering, including papers published in the early '80s that originated the term and practice of object-oriented design. Read his blog, Software architecture, software engineering, and Renaissance Jazz.
Ali Arsanjani
Ali Arsanjani, Ph.D., is chief architect for the SOA and Web Services Center of Excellence within IBM Global Services, specializing in harvesting and developing best practices for the modeling, analysis, design and implementation of SOA and Web services. He leads the internal IBM worldwide SOA and Web Services Community of Practice (4000 members) and is the principal author of the (Service-Oriented Modeling and Architecture) SOMA method for SOA. He is currently focusing on SOA Tooling with support for Modeling (SOMA), Assessments, Strategy and Planning, Governance, Architecture and Realization and their practical application to engagements inside and outside of IBM. Read his blog, Best Practices in Service Oriented Architecture.
Resources
About the author
Rate this page
| |
