Lessons From Continuous Software Delivery - InformationWeek

InformationWeek is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Healthcare // Policy & Regulation
11:30 AM
Anders Wallgren
Anders Wallgren
Connect Directly
50% Lessons From Continuous Software Delivery

A tight cycle of development, design, and testing is the best way to keep a big project (even a supposedly Agile one) from blowing up., President Obama's healthcare exchange website, has been heavily scrutinized this past year. In their quest to extend healthcare to every American, federal leaders put too many roadblocks and a tight deadline in the way of delivering a site that functioned and performed well during the initial rollout.

Like conventional manufacturing, software development is no easy feat, and the complexity is only continuing to increase. While the team of contractors who were hired to develop the site followed the principles of Agile, a rapid and lean software development process, it was not enough. In fact, neither the principles of Agile nor the succeeding practice of continuous integration could solve the software delivery challenge by themselves. What could help is operating in a continuous delivery model, which is described by many as the culmination of all of these development practices.

Continuous delivery is a software strategy that seeks to err on the side of delivering working software, as opposed to new features. The core idea is to create a repeatable, reliable, and iterative process by which working software is ushered from concept to customer. The foundation for this repeatable and reliable process usually means replacing manual, error-prone processes with automation.

[Users know what they want -- can you deliver? See Healthcare Software Development: We Can Do Better.]

The truth is that the journey to deliver a well-functioning, high-performance site is difficult, and the only way to keep up with increasing user demands is to evolve our software development processes to include these high levels of automation. There are two areas specifically that made a project difficult to pull off in the short time frame.

1. Too many control points
There are too many points of command and control across the application development lifecycle that do not offer visibility into who is doing what and when. The US government had about 55 contractors working on the site. They operated under a tight timeline and communicated with multiple agencies throughout the process. With critical stakeholders trying to make decisions under a timeframe that was less than suitable, it no doubt created major communication challenges. Teams need to be able to operate independently, and collaboratively, at a level that offers a continuous delivery of software updates.

2. Inadequate testing
Engineers who worked on fixing the site said there were stray lines of code that didn't seem to have a purpose. They found a lack of basic Web efficiency techniques and a failure to ensure the site could handle the website traffic load. If these issues were discovered and fixed ahead of the site moving into production, it could have saved the government a week's worth of distractions, a lot of embarrassment and political capital, and possibly tens of millions of taxpayers' dollars.

Automating as much of the testing process as possible and testing early and often are the keys to any software development project. Eliminating manual testing processes provides more immediate feedback to busy developers, allowing them to fix things rapidly. This accelerates cycle time and, as more features are added to the site, helps to verify the functionality of those changes and features more quickly. When activities like testing are not automated, the result is a labor-intensive process that can be time consuming and costly.

The site project underscores the complexity of building software today. While the project had many -- including the president -- scrambling for a fix, the extensive problems could have been addressed faster if a continuous delivery model was in place that combined Agile development, continuous integration, and DevOps. Operating in this model allows organizations to respond quicker to the end user to meet expectations. The larger the organization, the more important it will be to have this type of model in place.

According to an engineer on the project, the website contains about 500 million lines of software code. By comparison, a smartcar that connects to several systems has about 300 million lines of software code. Based on the number of lines of code in the project, there is no doubt that complexity of the system required good planning with clear, attainable goals from the start. Outlining those broader goals, as well as defining short-term milestones, is the key to any software development project today. The connected era that we live in requires real-time feedback and response, and the only way to get ahead is to measure progress and improve over time.

The debacle is arguably the most public software failure of 2013, and it sends a clear message that in today's demanding market software development must evolve into a modern approach. The movement to Agile was slow to move into the mainstream; over the past few years, organizations have embraced practices such as continuous integration to shorten cycle times. To compete in today's market, organizations need a fundamental approach that is a culmination of them all -- continuous delivery.

It doesn't matter whether your e-commerce D-Day is Black Friday, tax day, or some random Thursday when a post goes viral. Your websites need to be ready. Get the new Battle-Tested Websites issue of InformationWeek Tech Digest today. (Free registration required.)

Anders Wallgren brings with him more than 15 years of in-depth experience designing and building commercial software. Prior to joining Electric Cloud, he held executive positions at Aceva, Archistra, and Impresse. He also held management positions at Macromedia, Common Ground ... View Full Bio
We welcome your comments on this topic on our social media channels, or [contact us directly] with questions about the site.
Comment  | 
Print  | 
More Insights
Newest First  |  Oldest First  |  Threaded View
User Rank: Apprentice
11/9/2014 | 9:39:31 PM initiative was not "Agile"
I would challenge anyone to present evidence that the initiative was operating in even a basic Agile manner.  When you leave the core testing to the last 2-3 weeks before launch (I believe), you are not following even basic Agile tenets.

Agile itself emphasizes working software.  If nothing else, just read the Agile Manifesto, taking care not to overlook its Twelve Principles.  Here are three relevant ones:
  • Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.
  • Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.
  • Working software is the primary measure of progress.

Notwithstanding, being Agile is not easy, and the Manifesto does not implement itself.  I definitely agree that a Continuous Delivery (CD) approach (i.e., based on automation) is the best practical way of achieving agility, and would have helped mitigate a lot of the risk.  IMO, CD provides fundamental methodology and tools for the future of software engineering.  Because software is eating the world (per Marc Andreessen), those who practice CD will out-compete those who don't.
User Rank: Ninja
10/14/2014 | 1:23:04 PM
Too many cooks spoil the broth
.... especially when the meal is assembled at the dining room table, and not in the kitchen where it should have been.
Charlie Babcock
Charlie Babcock,
User Rank: Author
10/13/2014 | 4:22:41 PM
A DevOps approach would have been doomed needed agile development, continuous delivery and continuous integration, operating in a more of a DevOps approach. Easy to say this. But with so many parties involved, just about impossible to do.
What Becomes of CFOs During Digital Transformation?
Joao-Pierre S. Ruth, Senior Writer,  2/4/2020
Fighting the Coronavirus with Analytics and GIS
Jessica Davis, Senior Editor, Enterprise Apps,  2/3/2020
IT Careers: 10 Job Skills in High Demand This Year
Cynthia Harvey, Freelance Journalist, InformationWeek,  2/3/2020
White Papers
Register for InformationWeek Newsletters
Current Issue
IT 2020: A Look Ahead
Are you ready for the critical changes that will occur in 2020? We've compiled editor insights from the best of our network (Dark Reading, Data Center Knowledge, InformationWeek, ITPro Today and Network Computing) to deliver to you a look at the trends, technologies, and threats that are emerging in the coming year. Download it today!
Flash Poll