How DevOps Can Cut Innovation-Crushing Technical Debt

A good friend of mine is an awesome coder, but lousy with personal finances. Some years ago I remember him being hounded by creditors after he'd racked up a fairly sizeable credit card bill. It got so bad that he'd turn off his cell phone to avoid calls. Not good, you say? True, but at least these debt collectors weren’t in the "pay up or we’ll organize a hospital visit for you" type of game.

Geek nostalgia aside, the trials and tribulations of my credit challenged friend is analogous to the larger problems we face within enterprise IT when acquiring and failing to pay back what we commonly call "Technical Debt."

Coined originally in the Agile Manifesto back in 2001, technical debt refers to the price organizations pay when releasing badly designed code. Like my friend's problem, the more debt we accumulate, the more interest we pay later. When it comes to bad code, companies can accumulate so much technical debt that in severe situations any thought of innovation takes a back seat to costly firefighting.

Organizationally, the main blame falls on developers and architects. After all, any undetected code defect that makes its way into production is probably a 100 times more costly to fix than if it had been picked up in the early stages of development. And, as developers move onto other projects and priorities shift, defects become harder to fix and the compound interest grows. It’s a vicious cycle.

With increased business pressure to rapidly deliver software innovations, how can we avoid going into technical debt? The DevOps movement with its heritage in Lean thinking and Agile provides some of the answers.

Here are my top DevOps practices that help reduce technical debt.

1. Avoid bad spending habits – My coder friend with his credit card was like a kid in a candy store. He just couldn’t resist the urge to spend big, while avoiding payment until his next salary check. In IT we have similar bad habits we're allowed to get away with. For example if there are constant delays in release and deployment processes, a maxed out developer can perhaps "put off" some code optimization, knowing they can get back to it later since the code won't be released for a while anyway. Invariably they never get back to the fixes.

To combat these bad habits, teams should seek out and destroy the conditions that cause them. For example, to help prevent defects, teams can establish testing earlier in the software lifecycle. That situation and automated deployment processes mean teams can't hide sloppy code behind delays and release bottlenecks.

2. Consolidate your debt -- People with credit problems often spread the debt across many cards. In IT, our debt often spans multiple systems, many of which are built upon badly designed and developed code bases. In these cases it's no surprise that short-term fixes and bad code perpetuate themselves. If code is bad to begin what’s the point in finding a more elegant solution?

There's no easy answer to this problem, so my advice is to start an initiative to identify the biggest problems and address these first. This is a tough call, because it involves balancing new app development versus paying off our debt legacies. In the long term, however, the benefits should outweigh short-term slower delivery -- provided of course the design and architectural mistakes of the past aren’t repeated.

 3. Seek guidance and counselling -- Breaking bad personal finance habits is tough, so people normally seek outside help. In software development this involves using code- and peer-level reviews, mentorship, and management guidance. Too often, however, this advice centers entirely on development -- meaning IT dismisses operational considerations or, worse still, avoids them completely until the "debt" goes over the wall into production.

To avoid these situations, it's imperative to involve IT operations right from the start of development and continue to provide essential feedback (such as performance management guidance) across the software development lifecycle. This is where a DevOps approach helps, since a key aspect of the movement is closer collaboration and communication between development and operations teams.

4. Avoid "spend more" incentives -- Folks get into debt because they succumb to temptation -- like, spend today and earn double frequent flyer points, or enjoy 0% interest terms for six months (then get slugged). Sadly we often apply the same "carrots" in development -- using incentives tied to function points, the number of user stories delivered, or even lines of code written by a developer.

This approach might guarantee lots of code, but what about the usability? More functions might earn development a green light, but will the customer love those functions? Furthermore, if developers have incentives for just shipping code, why should they care if an application has a less-than-optimal database call, or doesn't immediately meet a security compliance requirement?

5. Think broadly about your debt position -- While it's easy to blame development for the accumulation of technical debt, operations can be just as guilty. For example, failing to document infrastructure services could lead to longer problem-diagnosis times, while poor insight into application performance and capacity during pre-production may lead to a knee-jerk procurement of unnecessary hardware.

Once again DevOps-style collaboration can help. By involving the development team in ongoing debt-reduction programs, DevOps teams can better identify people, process, and technology inefficiencies. For example, collectively reviewing mobile app analytics for usage patterns could help operations determine how poor network configurations are increasing latency and, most important, dragging down the customer experience.

In an age where customers call the shots, ignoring technical debt is deadly. Ultimately it ends up consuming much of the organizational blood, sweat, and tears that should be dedicated to digital innovation and business transformation.