How I Pulled Myself Out of the Waterfall - InformationWeek

InformationWeek is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

09:00 AM
Mark Thiele
Mark Thiele
Connect Directly

How I Pulled Myself Out of the Waterfall

Mark Thiele hates waterfalls. Not the ones found in nature, of course, but the traditional, sequential method of software development known as the waterfall model, in which a project flows downhill from one group to another in slow, rigid stages.

Imagine you’re having a house built. You’d assume the architect is working closely with the builder, and that both are staying in touch with you to make sure your requirements are being met. Once the foundation is laid and the house framed, you’d imagine that contractors would perform various tasks -- exterior siding, drywall, electrical work, plumbing, etc. – simultaneously rather than waiting for each phase to be completed before the next can begin. You expect the house will be completed on time, on budget and with minimal defects.

Software development projects should move the same way, but a surprising number of businesses are still bogged down by the waterfall method that takes a cumbersome, linear approach to the phases of conception, design and specification, development, testing, deployment, and maintenance. This antiquated system is why so many large software projects historically have run behind schedule and over budget, and have failed to meet the client’s rapidly evolving needs.

I was an IT executive several years ago at a large biotech company that was drowning in the waterfall method and our team knew it had to stop. We instituted an early experiment in DevOps, an organizational construct that calls for software developers and operations teams to exit their silos and collaborate closely to accelerate the frequency of code releases into production.

More than a buzzword

The experience vividly illustrated why DevOps has become such a hot industry buzzword: because it really works.

The biotech firm’s waterfall development process was a tedious mess.

Application teams often worked without direct contact with operations. Any application change required a review of company and US Food and Drug Administration policies. This sometimes occurred in Dev and sometimes in Ops.

Server and storage requirements were handled by different operations teams with different schedules.

Only after the storage and server provisioning was complete would the network and security teams be brought in.

At the end of these steps, all the parties would be brought together for additional reviews.

The result: Software projects were often delayed by months, costing the company millions of dollars in lost revenue opportunities, not to mention the late delivery of product features that ultimately could help patient treatment.

Benefits: Time, money, quality care

Enter a more agile philosophy. Our first move was to increase collaboration among the development, operations and compliance teams at the beginning of a project. This alone cut two weeks off the average release-to-production timetable under the waterfall method.

We then consolidated information about infrastructure capacity delivery to remove the roadblock of that information being available only to certain individuals on isolated teams.

We simplified major infrastructure capacity delivery models to allow for more rapid allocation of capacity across network, storage, and virtualized servers.

We shed the company’s habit of lumping together multiple updates into one larger release (a defense mechanism against the slow, ponderous process for deploying individual updates). The new, faster process we put in place allowed for the release of smaller updates, which in turn, reduced the number of application, infrastructure failures and performance issues that can accompany a single large release.

We instituted uniform compliance and security requirements for most application changes, rather than the previous hodgepodge.

All of these changes led to a huge reduction in the company’s average release-to-production cycle, from eight weeks or more to less than three weeks. We improved customer satisfaction and gained back some lost revenue opportunity.

[Read Mark Thiele's recent commentary in InformationWeek, The Strategy of Speed in IT.]

As this experience showed, the waterfall no longer makes sense for the delivery of software-based projects where there is a benefit to quickly responding to customer needs and changing market conditions.

If you’re still in it, you need to pull yourself out of the waterfall too.


Mark Thiele's successful career in IT spans 25 years and has focused on both operating roles and on driving cloud adoption across enterprises of all sizes. Mark has deep industry experience and extensive knowledge of the requirements of policy-driven cloud computing and ... View Full Bio
We welcome your comments on this topic on our social media channels, or [contact us directly] with questions about the site.
Comment  | 
Print  | 
More Insights
Newest First  |  Oldest First  |  Threaded View
User Rank: Apprentice
9/20/2018 | 2:09:48 AM
Pending Review
This comment is waiting for review by our moderators.
User Rank: Ninja
3/11/2017 | 8:46:03 AM
There is always a waterfall...ideally
As the article states in the example, for a contractor to put in plumbing and nail siding to the outside walls the foundation and framing have to be in place. That IS waterfall!! And there is really no way around it. In software development QA cannot test something that does not exist, be it the requirements (which in the real agile world are often not specified) or the code or the database schema.

There is always an element of waterfall in the process and keeping some aspects is beneficial. How many times did I have to do work on a high priority feature that in all agility was pushed from nowhere straight to the top of the backlogg just to be considered not that important after all when we found out that due to lack of requirements the implementation became very difficult, took a long time, and had to be constantly changed?

Be more waterfallish at the beginning and keep a stricter first in, first out approach, then follow up with a healthy dose of analytics and up front design. After that be as agile as can be with writing tests, docs, and code in pararallel followed by testing and bug fixing at the same time.

Sure, you can pour the foundation and build the roof at the same time in the same place, but it is tremendously more difficult and the end result will not be as good as doing it one after the other.
User Rank: Moderator
3/8/2017 | 5:01:11 AM
Your article gave me many interesting ideas for my presentation. super!
Charlie Babcock
Charlie Babcock,
User Rank: Author
3/1/2017 | 1:59:19 PM
Frequent updates, continuous integration the real goal
Author Mark Thiele has hit upon the key aspects of waterfall development that DevOps is meant to counteract. Commenter Mark Sitkowski is probably right in saying waterfall isn't always as bad as the example cited. But in general, a more rapid development/more frequent update process must displace today's existing software production processes, even when they work. The problem isn't just getting the project done but getting the application changes into production when they're needed. Most companies have found it to be a major culture change. 
User Rank: Strategist
2/28/2017 | 5:55:08 PM
I must have been to the wrong school, or something, but the process you've described isn't the waterfall I'm used to - especially the bit about everyone waiting for everyone else to finish. While you're still at the planning stage, there should be meetings with all the developers and testers, to determine

a) what discrete tasks need to be performed,

b) which tasks depend on which other tasks and

c) which tasks can be performed in parallel with which tasks (which is rather determined by (b), above).

If you have enough resources, you just allocate one task each, and join your work flows at the dependency points. The longest path through the plan determines the project duration.

I don't care what methodology you use, putting the roof on before the walls are built won't work, which is why things called 'critical paths' exist. 

Also, any PM who doesn't hold regular meetings (or 'standups' if you like) so that development groups can exchange information, is in the wrong job.
10 Ways to Transition Traditional IT Talent to Cloud Talent
Lisa Morgan, Freelance Writer,  11/23/2020
What Comes Next for the COVID-19 Computing Consortium
Joao-Pierre S. Ruth, Senior Writer,  11/24/2020
Top 10 Data and Analytics Trends for 2021
Jessica Davis, Senior Editor, Enterprise Apps,  11/13/2020
White Papers
Register for InformationWeek Newsletters
The State of Cloud Computing - Fall 2020
The State of Cloud Computing - Fall 2020
Download this report to compare how cloud usage and spending patterns have changed in 2020, and how respondents think they'll evolve over the next two years.
Current Issue
Why Chatbots Are So Popular Right Now
In this IT Trend Report, you will learn more about why chatbots are gaining traction within businesses, particularly while a pandemic is impacting the world.
Flash Poll