The Fall of Waterfall

Does your development methodology send you down Niagara Falls in a barrel?

For all the advantages of using an alternative approach, however, many organizations are resisting changing their deficient existing processes. The primary concerns that keep organizations from embracing these new methodologies are:

  • Logistics of stakeholder interaction. EXtreme Programming, by far the most well known of these methodologies, requires that the business stakeholders be in close proximity to the system developers so there can be interaction on a continual basis to prioritize new features and provide feedback. In general, assuming that the business stakeholder has a primary job outside of the project, this interaction is not always feasible.
  • Logistics of IT resource allocation. Many organizations have very specialized IT roles (such as a data modeler who works with the team during the design phase), and it would be difficult to allocate the diverse set of necessary resources with an iterative schedule where there involvement would be needed much more frequently.
  • Financial objections. Business users and financial analysts like to know what they are getting, and for how much. The iterative approach's quintessential response to this question is "you're going to get as many of the high-priority features that we are able to do in the timeframe and budget that are available to us." Needless to say, this usually doesn't go over very well with the CFO. The fact that these investments usually require ROI justification makes the financial objections even more germane. Additionally, a large project can be capitalized, meaning that the business can pay for it as a one-time expenditure and in most cases depreciate the solution's cost over time with less of a negative effect on short-term profits. In some cases, the expenses associated with a full-time staff that continuously develop a solution in an iterative fashion would be difficult to capitalize in the same manner.
  • Perception issues. Many of the issues that have tainted the use of iterative methodologies relate to perception. Many still believe that "eXtreme programmers" are college dropouts who say "dude" a lot and can't be counted on to be at work during high tide. However, these prejudices often lead to "It isn't for our company" dismissals without truly understanding the methodology's issues and benefits.
  • Effort required. The iterative approach requires a significant amount of effort and management because the timeframes are shorter. If a developer is out for a week during a four-week iteration, it is a big loss to the project. Many feel that the iterative approach is too demanding because it focuses on consistent workload rather than workloads with very high variability.
  • Cost of change. Finally, those who are convinced of the merits of agile methodologies typically quiver at the idea of trying to justify the cost and effort involved to make the transition. These hard and soft costs often include new training, new tools, different resource allocations, resource changes, and the loss of productivity during the transition.

How We Can Improve

I believe that for most organizations looking to use alternative methodologies, a controlled evolution is the best mechanism for a transition to an agile methodology. To alleviate some of the short-term issues with the waterfall methodology and to assist in a migration toward an agile approach, I recommend:

  • Reducing projects' scope and team size to about one half of your current larger projects. Break up larger efforts into multiple projects.
  • Allocating 20 to 30 percent of the project budget to perform post release refactorings, stabilization, improvements, developer training, and documentation.
  • Budgeting like waterfall but developing in an iterative fashion. A good friend suggested this idea, which works well to allow projects to be prebudgeted, but still benefit from an iterative approach.
  • Doing more interim releases and demos in an iterative fashion. Enhance your ability to get early feedback and raise your team's confidence in meeting deadlines by releasing on a frequent basis. Much like completing a project, the most satisfying culmination to an iteration is in deployment — even if it's to only a subset of users.
  • Staffing the projects with dedicated resources. Resources allocated as part time contribute little and often detract from progress.

If you build systems that, like bridges, have only one primary purpose or feature, the waterfall methodology should be fine. But if you're looking to make your systems do things that bridges can't, like increase feature sets every six to eight weeks and cost less to maintain over time, it's time to upgrade your methodology.

Robert Northrop [[email protected]] is a director of design and development with Tallan, a professional services company specializing in developing custom technology solutions for its clients.

Editor's Choice
Samuel Greengard, Contributing Reporter
Cynthia Harvey, Freelance Journalist, InformationWeek
Carrie Pallardy, Contributing Reporter
John Edwards, Technology Journalist & Author
Astrid Gobardhan, Data Privacy Officer, VFS Global
Sara Peters, Editor-in-Chief, InformationWeek / Network Computing