Automation for the people: Continuous feedback

Document options Document options requiring JavaScript are not displayed Sample code Rate this page Help us improve this content

Level: Introductory

Paul Duvall (paul.duvall@stelligent.com), CTO, Stelligent Incorporated

14 Nov 2006

Feedback is vital for the practice of Continuous Integration (CI) -- in fact, it's the life blood of a CI system. Rapid feedback enables speedy responses to build events that require attention. Without feedback mediums like e-mail or RSS, builds in a broken state have the tendency to stay broken, which defeats the purpose of CI in the first place! In this installment of Automation for the people, automation expert Paul Duvall examines various feedback mechanisms that you can incorporate into CI systems.

When I describe CI to technologists that haven't used the practice yet, I tend to focus on one of its major benefits: a decrease in the time between when a defect is introduced and when it is fixed. This benefit is a direct result of a CI server's ability to provide rapid feedback of build statuses. In my opinion, rapid feedback of build events is the central tenet of CI and one that causes many to describe a CI system as a continuous feedback mechanism.

When a build is broken, I want to know immediately so I can take the proper action to fix it -- whether that be a compilation error, test failure, or even an increase in overall complexity. The sooner I know, the more relevant the information and the better chance I have to fix the problem.

Feedback, however, is useless without action. In the case of CI, this action must be prompt because a broken build affects everyone; therefore, the feedback mechanism employed by a CI server must be timely. The beauty is that most, if not all, CI servers available today provide an e-mail feedback mechanism; however, there is a cornucopia of other mediums to choose from ranging from Really Simple Syndication (RSS) feeds to SMS text messages to X10 devices. And these various feedback mechanisms quickly facilitate jumpstarting the necessary people into action.

About this series As developers, we work to automate processes for users; yet, many of us overlook opportunities to automate our own development processes. To that end, Automation for the people is a series of articles dedicated to exploring the practical uses of automating software development processes and teaching you when and how to apply automation successfully.

Execution a la e-mail

The ubiquitous nature of e-mail makes it a simple and useful means of feedback -- every CI server I'm familiar with provides some type of e-mail feedback mechanism. In fact, most CI servers provide the capability for configuring who receives which message, based on a desired build event. For instance, a technical lead may always receive an e-mail regardless of build status, but the project manager only receives an e-mail when the build fails. Additionally, the developer or developers who checked any code into a CM system (and thus triggered a CI build) always receive an e-mail indicating the build's status.

Listing 1 is an example of a CruiseControl main configuration file (config.xml), which is configured to send an e-mail, in HTML format, to the project's "build master" for both successful and failed builds:

<publishers>
  <htmlemail
    mailhost="localhost" 
    defaultsuffix="localhost"
    username="bfranklin"
    password="G0Fly@Kite"
    returnname="Buildmaster"
    returnaddress="buildmaster@localhost"
    subjectprefix="[CC]"
    xsldir="webapps/cruisecontrol/xsl"
    css="webapps/cruisecontrol/css/cruisecontrol.css">
    <always address="buildmaster@localhost"/>
  </htmlemail>
<publishers>

You can use the defaultsuffix attribute of CruiseControl's e-mail task to also send e-mails to the developer(s) who committed the most recent changes, as well.

Figure 1 shows an example of the e-mail a user will receive based on the configuration in Listing 1:

While e-mail is arguably the de facto standard feedback mechanism for projects that use CI, if team members are flooded with too many e-mails, they may begin to ignore them, which, of course, defeats the purpose of a feedback mechanism in the first place. I find it more productive to configure CI systems to send e-mails only to the developer or developers who applied the most recent change to a code base and to project leads who don't mind receiving a flood of build status e-mails every day.

What is Continuous Integration? Continuous integration (CI) is a practice that enables teams to receive feedback and make improvements on a continual basis rather than waiting until later in the development cycle to find and fix defects. Tools (like CruiseControl) run in the background polling a version control repository looking for changes. When a change does occur, the tool executes a predefined build script through Ant, for example. Continuous Inspection is facilitated by the practice of continuous integration

RSS to the rescue

RSS is both a passive and active form of feedback, which can make this feedback mechanism more appealing because of a reduction in the amount of e-mail messages you'd receive otherwise. An RSS feed is a generated XML file (which conforms to a standard) that is updated based on certain events. An RSS reader then picks up the change and creates a passive message indicating new content. So, if you'd rather not get bombarded with e-mail, try this approach instead.

Within a CI system, this RSS feed can be updated based on the latest build status. For instance, Figure 2 is an example of the CruiseControl RSS XML that, by default, is generated for any build event:

Simply point your handy RSS reader to this feed and when a new build event occurs, and the RSS feed is updated, bingo, you'll be notified. Figure 3 shows the RSS feed as displayed in an RSS reader:

Back to top

Stir it up with SMS

What if you're away from your computer and the build fails? If you believe that keeping an integration build successful is one of the highest priorities on your project, you'll want to know the build status at any point in time. One way to constantly keep abreast of build statuses is through the Short Message Service (SMS), which are short text messages sent to mobile phones.

Fortunately, the same mechanism used to send e-mail can also be used to send SMS text messages. For example, Listing 2 shows an example of using the CruiseControl e-mail task to send an SMS message:

<publishers>
  <email mailhost="localhost" 
    returnaddress="buildmaster@localhost"
    returnname="Buildmaster"
    reportsuccess="fixes"
    subjectprefix="brewery"
    buildresultsurl="">
  <failure address="7035551212@vtext.com"/>
  </email>
</publishers>

Why so many feedback mechanisms? When using CI, the criticality of a broken build is paramount and many teams do not move to other tasks until the integration build is successful again. Accordingly, it's often effective to choose multiple mechanisms that serve different purposes, such as SMS for those who aren't at their computers often and X10 devices that enable people to know a build's status in a glance. It is probably ineffective to use all the feedback mechanisms I cover here on a single project-- choose what works best for you and change it up every now and again to add some spice to a project, so to speak.

I have configured the e-mail task in Listing 2 in such a manner that I will receive text messages only when there is a build failure, which is an effective means to reducing SMS messages. Furthermore, the reportsuccess="fixes" attribute of the e-mail task indicates that you will receive a follow-up text message when the build has been fixed.

Figure 4 shows an example of the message sent from the CruiseControl server when a build fails:

Using SMS to send text messages can be an effective form of feedback for urgent build status messages. Generally, it is not recommended that you send text messages every time a build occurs (especially if you pay per message!). Remember, too much information can become "noise" that reduces the effectiveness of feedback, so choose wisely.

Back to top

Excite things with X10

E-mails work well and SMS is great when you're away from the office, but what if you just want to glance at something to determine a build's status? Luckily, there is an entire cottage industry around programmable devices, dubbed X10 devices, which can provide a more passive form of feedback when hooked up to CI systems. With X10-enabled devices, you can control most any electrical device that can be enabled using special hardware that can be manipulated through the X10 protocol. This means that every day devices such as lights can be used to notify us of a build failure.

For instance, Listing 3 offers an example of the code required to plug an X10 device into CruiseControl's publisher mechanism. This example demonstrates turning on a red warning light if the build fails. The light is connected to an X10 controller (several "X10 kits" are commercially available). CruiseControl includes the Java™ COMM API so that you can use CruiseControl's config.xml to communicate with the device and turn it on or off. The X10 controller will connect to a communications port on your build machine; the port attribute indicates the communications port you are using. The houseCode and deviceCode attributes refer to the letter and number you set on the X10 controller for the device. The onWhenBroken attribute indicates that the device will turn on when the build fails (and off when successful). Finally, the interfaceModel attribute refers to X10 computer interface. The default is CM11A.

<publishers>
  <x10 port="COM1" houseCode="A"  
  deviceCode="3" onWhenBroken="true" interfaceModel="CM17A"/> 
</publishers>

Now, when this publisher is utilized, a device, such as the one shown in Figure 5, will turn on when the build is broken:

There are a variety of X10 devices -- LavaLamps, sirens, you name it -- that can really enliven a project's environment. Adding these devices to a room is bound to create excitement, along with a clear indication of a build's status.

Back to top

Don't forget about monitors and IM

If you'd rather receive feedback instantly, have a look at client monitors and instant messaging. A client monitor is an application that is always running in the background and periodically polls your build server for the latest build status. Of course, everyone knows what instant messaging is, and a host of popular platforms, from AIM to Yahoo to MSN, are available for this purpose.

Monitoring status with monitors

CruiseControl.NET (a Continuous Integration server for the .NET platform) provides a monitoring mechanism called CCTray that works with both CruiseControl.NET and the traditional Java built CruiseControl and is run from the Windows taskbar (which means it works only on Windows machines). An example of CCTray in action is shown in Figure 6:

You can use another client monitor called Nag on non-Windows machines and even on other CI servers, such as Apache Continuum.

Instant feedback with IM

You can use instant messages (or IMs) to notify project members of the status of the build too. Just like using a monitor, it's another way to receive feedback quickly because of IM's ubiquity among developers.

CI servers like Apache Continuum provide a simple way to send IMs regarding a build's status. For example, Figure 7 displays an example of a failed build message, from Continuum, which is sent to a Google Talk instant message client using Jabber. CruiseControl also provides a Jabber publisher to send messages.

Monitors and IMs are some of the quickest ways to send build status feedback, and I'm familiar with project teams that exclusively use these mechanisms. As long as this is coupled with another way to inform project members when they are away from their desk (such as with e-mail or SMS), this can be a highly effective form of feedback.

Back to top

A party of feedback mechanisms

As you can see thus far, there are many types of feedback mechanisms available for you to utilize in a CI environment. Many other feedback mechanisms, which I did not cover, are available, including the Ambient Orb, Web browser plug-ins, sounds, and widgets. Table 1 provides a summary of the feedback mechanisms I covered in this article, plus a few others I didn't:

The most effective way to proactively generate action is to use multiple feedback mechanisms in concert. Choose the combination that's most appropriate for you.

Back to top

Have fun with it!

Remember, the purpose of build feedback is to initiate rapid action so that teams can fix a broken build. Typically, you'll need a combination of these feedback mechanisms to elicit this action. For instance, you may provide e-mails for all build events, SMS for build failures (and follow-up fixes), and an X10 device for visual notification. As your team matures in using these devices, you'll want to change it up every so often to add variety and keep people interested in build statuses. Be creative and have fun with it -- it's amazing how adding a few feedback gimmicks can enliven the mood and generate interest in the information they are providing!

Back to top

Download

	Information about download methods

Resources

• Automation for the people (Paul Duvall, developerWorks): Read the complete series.
  
  
• "Bubble, Bubble, Build's In Trouble": Mike Clark, author of Pragmatic Project Automation, demonstrates a detailed example of configuring an X10 device for CruiseControl.
  
  
• "Jabber" (Gerhard Poul, developerWorks, May 2002): Gerhard Poul shows how XML-based Jabber fits into today's e-business infrastructure, highlighting instant messaging in a whole new way.
  
  
• "eXtreme Feedback for Software Development" (Alberto Savoia, DeveloperTesting, April 2004): Other examples of Continuous Feedback Mechanisms -- or as the author describes them: eXtreme Feeback Devices.
  
  
• "Pragmatic Automation": Mike Clark, author of Pragmatic Project Automation, covers various feedback devices to monitor build status.
  
  
• "Continuous Integration" (Martin Fowler, martinfowler.com, May 2006): A description of the purpose and practices of Continuous Integration.
  
  
• "Configuring Continuum for CI" (Testearly.com): An article that describes the basic setup and configuration of Continuum.
  
  
• Ambient Orb Ant task example (Testearly.com): Examples on using the Ambient Orb Ant task.
  
  
• The Java technology zone: Hundreds of articles about every aspect of Java programming.
  
  

• CruiseControl: An open source server for Continuous Integration.
  
  
• 4-Pack Firecracker Automation System: Hardware to help configure your build to interface with X10 devices.
  
  
• Nag: A Continuous Integration montior that works with multiple CI servers and platforms.
  
  
• Yahoo! Widgets: Search for CruiseControl widgets for feedback to your desktop.
  
  
• Ambient Orb Ant task: A freely available open source Ant task for changing Ambient Orb colors based on build status.
  
  

• Improve Your Code Quality discussion forum: Regular developerWorks contributor Andrew Glover brings his considerable expertise as a consultant focused on improving code quality to this moderated discussion forum.

About the author

		Paul Duvall is the CTO of Stelligent Incorporated, which helps companies address software quality with effective developer testing strategies and Continuous Integration techniques that enable teams to monitor and improve code quality early and often. He is a contributing author to the UML™ 2 Toolkit and currently co-authoring Continuous Integration: Improving Software Quality and Reducing Risk (Addison-Wesley).

Rate this page

Please take a moment to complete this form to help us better serve you. Did the information help you to achieve your goal? Yes No Don't know   Please provide us with comments to help improve this page:   How useful is the information? 1 2 3 4 5 Not useful Extremely useful

Back to top