Federal executives are usually reluctant to talk about their IT program failures -- especially when they involve big programs like the Air Force's Enterprise Combat Support System. But the failure of ECSS also offers valuable lessons to federal executives and CIOs who are trying to support them.
The idea behind ECSS was to implement an enterprise resource planning (ERP) system that would have "subsumed" more than 250 major existing data systems affecting more than 250,000 users. ECSS was to cost more than $5 billion and take about seven years to implement completely.
After losing altitude and airspeed for some time due to a variety of problems -- some unavoidable, some self-inflicted -- ECSS crashed and burned last year. I say "burned" because it's unclear how much of the work on that program may be used by the Air Force beyond the lessons that came out of it.
[ Agencies are gaining ground when it comes to automating information exchange. Read Information Sharing: Feds Cite Progress. ]
Based on my personal experience working in and on the Air Force supply chain for 26 years and inside the ECSS program on the contractor side for more than four years, I concluded the following factors were the major contributors to the program's failure:
Leadership and Change Management
The magnitude of the program made the Secretary of the Air Force the lowest level of leadership with full direct authority over all impacted organizations. But because ECSS was largely viewed as just a "logistics" system, when it came to oversight and involvement, the Air Force wasn't ready for the change management required and couldn't get ready in time for the program's survival.
The size and scope of ECCS were also too big for the Air Force to assimilate, based on complexity, number of systems and impacted organizations, pace of implementation and inability to deal with management risks.
The firm fixed price (FFP) acquisition proved too inflexible for the immaturity of the program requirements. That led to an over-administrated, adversial environment with many contract changes.
There were major requirements changes, which resulted in a loss of focus on the business case. The program's management was also hindered by politically influenced decisions and concerns over possible conflicts of interest.
Because of inadequate requirement definitions, it was very difficult to adopt industry-standard supply chain processes -- there was always a tendency to revert to legacy processes driven by legacy systems.
ECSS exposed a tremendous amount of data inaccuracy and unavailability that, despite heroic efforts, was not fixable to meet program requirements.
The Air Force selected ERP software applications before selecting the systems integrator. When the applications could not meet program requirements, it drove major cost increases and schedule delays.
ECSS was by far the largest business system implementation the Air Force had ever attempted for establishing application development environments and running on its network. Despite strong efforts to anticipate, find and fix bottlenecks, ECSS exposed numerous problems that couldn't be overcome.
Post Mortem: Cleaning up the Crash Site
What can a federal CIO and agency executives learn from ECSS and other IT crashes – especially in trying to move forward after a major failure?
First, get painfully real about the fundamental causes of the crash, not what people might be saying. Be careful not to "drink your own bath water," and focus on the things that must be fixed before you move on.
As for the Air Force example and a potential "son of ECSS," it's not happening. However, all the reasons the Air Force needs to modernize and transform its legacy logistics data systems remain. And now there are the added pressures of Defense Department budget cuts, sequestration and the Congressional mandate to achieve financial "audit readiness" by 2016.
Envision the end state and set a course (with flexible options over time). There was nothing wrong with the ECSS vision and its assessment of problems to be fixed and opportunities to be pursued. The bigger the program and the change, however, the bigger the leadership requirement. And if it's a big change, the CEO must lead -- leadership can't be totally delegated to the CIO. Additionally, the CIO needs to serve as the chief integration officer. To paraphrase Tiger Woods, don't take a golf shot if you don't commit to it 100%.
Set the criteria for how to chunk portions of the work -- by enabling implementation iterations, for instance -- much the same as in an Agile software development. The idea is to create a process that will achieve the vision based upon scope factors that include the following:
-- Number, complexity and size of processes/systems.
-- Mission (business) impact of processes/systems.
-- Costs versus benefits of the chunks, or iterations.
-- Change management difficulty (from users, leaders, and incumbent contractors to Congress).
-- Timing relative to other affected processes/systems/organizations (e.g. changing politics/missions that drive changes in priorities).
Use a hybrid contract strategy, with firm fixed pricing where requirements are nailed down and other types of contracts or task orders where the requirements aren't certain. Use a structured methodology, but don't let it use you. In the end, find the balance between true risk and reward for the scope and speed of the implementation.
Less is more, especially in the sense that government program management cannot become bureaucratic. Strict, 100% completion "waterfall" methodologies for major process reengineering and associated IT development will shoot down the best program. Lack of speed will also kill a program, as it did for ECSS. The entire government or industry team must clearly see that they are on the same flight.
Core mission or business capabilities requirements must drive the processes that are enabled by the IT implementation. They must also meet compliance requirements along the way, not just in the final tests. It's not about the software -- that's just a tool. Process automation isn't enough either; it is only part of the solution. An IT program's success is all about the mission/business value delivered by implementing improved processes. That means a holistic approach, in which documentation, training, etc., are given appropriate emphasis.
When ECSS crashed, it didn't take all the progress on data management with it. The Air Force learned a great deal about the importance of logistics data and the location of its authoritative sources. Those efforts must continue, regardless of the IT approach.
For enterprise-level, transformative IT programs, the CIO also needs to recognize that collaborating with the system integrator on strategy and tactics is critical to the program's success; technology and products take a back seat.
While IT infrastructure still needs significant improvements, implementing smaller "chunks" will make that process possible through continuous iterations. This isn't really a technology problem; it's more of about collaboration, how to use the technology, and the governance of standards, security and performance.
Federal CIOs can still meet mission requirements and capabilities and still create tremendous savings -- as in billions of dollars -- but only by learning these important lessons and reestablishing the confidence to get back on board and launch the needed modernization/transformation.
Bold visions are competing with practical budget realities for federal IT leaders. Our latest annual survey looks at the top IT priorities. Also in the new, all-digital Tech Priorities issue of InformationWeek Government: IT leaders are making progress improving the efficiencies in their IT operations, but many lack the tools to prove it. (Free registration required.)