How I Pulled Myself Out of the Waterfall - InformationWeek
IoT
IoT
DevOps
Commentary
2/28/2017
09:00 AM
Mark Thiele
Mark Thiele
Commentary
Connect Directly
Twitter
LinkedIn
Google+
RSS
50%
50%

How I Pulled Myself Out of the Waterfall

Mark Thiele hates waterfalls. Not the ones found in nature, of course, but the traditional, sequential method of software development known as the waterfall model, in which a project flows downhill from one group to another in slow, rigid stages.

Imagine you’re having a house built. You’d assume the architect is working closely with the builder, and that both are staying in touch with you to make sure your requirements are being met. Once the foundation is laid and the house framed, you’d imagine that contractors would perform various tasks -- exterior siding, drywall, electrical work, plumbing, etc. – simultaneously rather than waiting for each phase to be completed before the next can begin. You expect the house will be completed on time, on budget and with minimal defects.

Software development projects should move the same way, but a surprising number of businesses are still bogged down by the waterfall method that takes a cumbersome, linear approach to the phases of conception, design and specification, development, testing, deployment, and maintenance. This antiquated system is why so many large software projects historically have run behind schedule and over budget, and have failed to meet the client’s rapidly evolving needs.

I was an IT executive several years ago at a large biotech company that was drowning in the waterfall method and our team knew it had to stop. We instituted an early experiment in DevOps, an organizational construct that calls for software developers and operations teams to exit their silos and collaborate closely to accelerate the frequency of code releases into production.

More than a buzzword

The experience vividly illustrated why DevOps has become such a hot industry buzzword: because it really works.

The biotech firm’s waterfall development process was a tedious mess.

Application teams often worked without direct contact with operations. Any application change required a review of company and US Food and Drug Administration policies. This sometimes occurred in Dev and sometimes in Ops.

Server and storage requirements were handled by different operations teams with different schedules.

Only after the storage and server provisioning was complete would the network and security teams be brought in.

At the end of these steps, all the parties would be brought together for additional reviews.

The result: Software projects were often delayed by months, costing the company millions of dollars in lost revenue opportunities, not to mention the late delivery of product features that ultimately could help patient treatment.

Benefits: Time, money, quality care

Enter a more agile philosophy. Our first move was to increase collaboration among the development, operations and compliance teams at the beginning of a project. This alone cut two weeks off the average release-to-production timetable under the waterfall method.

We then consolidated information about infrastructure capacity delivery to remove the roadblock of that information being available only to certain individuals on isolated teams.

We simplified major infrastructure capacity delivery models to allow for more rapid allocation of capacity across network, storage, and virtualized servers.

We shed the company’s habit of lumping together multiple updates into one larger release (a defense mechanism against the slow, ponderous process for deploying individual updates). The new, faster process we put in place allowed for the release of smaller updates, which in turn, reduced the number of application, infrastructure failures and performance issues that can accompany a single large release.

We instituted uniform compliance and security requirements for most application changes, rather than the previous hodgepodge.

All of these changes led to a huge reduction in the company’s average release-to-production cycle, from eight weeks or more to less than three weeks. We improved customer satisfaction and gained back some lost revenue opportunity.

[Read Mark Thiele's recent commentary in InformationWeek, The Strategy of Speed in IT.]

As this experience showed, the waterfall no longer makes sense for the delivery of software-based projects where there is a benefit to quickly responding to customer needs and changing market conditions.

If you’re still in it, you need to pull yourself out of the waterfall too.

 

Mark Thiele's successful career in IT spans 25 years and has focused on both operating roles and on driving cloud adoption across enterprises of all sizes. Mark has deep industry experience and extensive knowledge of the requirements of policy-driven cloud computing and ... View Full Bio
Comment  | 
Print  | 
More Insights
Comments
Oldest First  |  Newest First  |  Threaded View
MarkSitkowski
50%
50%
MarkSitkowski,
User Rank: Strategist
2/28/2017 | 5:55:08 PM
Waterfall?
I must have been to the wrong school, or something, but the process you've described isn't the waterfall I'm used to - especially the bit about everyone waiting for everyone else to finish. While you're still at the planning stage, there should be meetings with all the developers and testers, to determine

a) what discrete tasks need to be performed,

b) which tasks depend on which other tasks and

c) which tasks can be performed in parallel with which tasks (which is rather determined by (b), above).

If you have enough resources, you just allocate one task each, and join your work flows at the dependency points. The longest path through the plan determines the project duration.

I don't care what methodology you use, putting the roof on before the walls are built won't work, which is why things called 'critical paths' exist. 

Also, any PM who doesn't hold regular meetings (or 'standups' if you like) so that development groups can exchange information, is in the wrong job.
Charlie Babcock
50%
50%
Charlie Babcock,
User Rank: Author
3/1/2017 | 1:59:19 PM
Frequent updates, continuous integration the real goal
Author Mark Thiele has hit upon the key aspects of waterfall development that DevOps is meant to counteract. Commenter Mark Sitkowski is probably right in saying waterfall isn't always as bad as the example cited. But in general, a more rapid development/more frequent update process must displace today's existing software production processes, even when they work. The problem isn't just getting the project done but getting the application changes into production when they're needed. Most companies have found it to be a major culture change. 
Shantaram
50%
50%
Shantaram,
User Rank: Moderator
3/8/2017 | 5:01:11 AM
Re: 192.168.1.1
Your article gave me many interesting ideas for my presentation. super!
moarsauce123
50%
50%
moarsauce123,
User Rank: Ninja
3/11/2017 | 8:46:03 AM
There is always a waterfall...ideally
As the article states in the example, for a contractor to put in plumbing and nail siding to the outside walls the foundation and framing have to be in place. That IS waterfall!! And there is really no way around it. In software development QA cannot test something that does not exist, be it the requirements (which in the real agile world are often not specified) or the code or the database schema.

There is always an element of waterfall in the process and keeping some aspects is beneficial. How many times did I have to do work on a high priority feature that in all agility was pushed from nowhere straight to the top of the backlogg just to be considered not that important after all when we found out that due to lack of requirements the implementation became very difficult, took a long time, and had to be constantly changed?

Be more waterfallish at the beginning and keep a stricter first in, first out approach, then follow up with a healthy dose of analytics and up front design. After that be as agile as can be with writing tests, docs, and code in pararallel followed by testing and bug fixing at the same time.

Sure, you can pour the foundation and build the roof at the same time in the same place, but it is tremendously more difficult and the end result will not be as good as doing it one after the other.
[Interop ITX 2017] State Of DevOps Report
[Interop ITX 2017] State Of DevOps Report
The DevOps movement brings application development and infrastructure operations together to increase efficiency and deploy applications more quickly. But embracing DevOps means making significant cultural, organizational, and technological changes. This research report will examine how and why IT organizations are adopting DevOps methodologies, the effects on their staff and processes, and the tools they are utilizing for the best results.
Register for InformationWeek Newsletters
White Papers
Current Issue
IT Strategies to Conquer the Cloud
Chances are your organization is adopting cloud computing in one way or another -- or in multiple ways. Understanding the skills you need and how cloud affects IT operations and networking will help you adapt.
Video
Slideshows
Twitter Feed
Sponsored Live Streaming Video
Everything You've Been Told About Mobility Is Wrong
Attend this video symposium with Sean Wisdom, Global Director of Mobility Solutions, and learn about how you can harness powerful new products to mobilize your business potential.
Flash Poll