Continuous Delivery: 7 Hard-Earned Lessons for Getting it Right - InformationWeek

InformationWeek is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

IoT
IoT
DevOps
Commentary
4/19/2017
04:00 PM
Russell Smith, CTO of Rainforest
Russell Smith, CTO of Rainforest
Commentary
0%
100%

Continuous Delivery: 7 Hard-Earned Lessons for Getting it Right

As organizations move toward continuous integration and delivery of applications there are multiple ways in which things can go wrong.

It seems laughable now, but do you remember when Internet Explorer was updated every two years? Times have changed. Development cycles are approaching warp speed as companies deal with the competitive pressure to update their applications and websites as frequently as possible.

Facebook has said it releases new code into production twice a day -- and that was three years ago. Google Chrome is fast approaching Version 60. Most businesses don’t need to release new features so frequently, but everyone wants to be quicker. A Forrester study found that 51 percent of business leaders want new software capabilities to go from concept to delivery in fewer than six months, but that most developer teams can’t accommodate that pace.

Organizations that need to move fast have adopted continuous integration and delivery as their model. With CI/CD, new features are released as soon as they’re ready instead of following some arbitrary release cycle. We’ve done continuous delivery at Rainforest for a few years now and I can attest to the benefits, but it’s tricky to get right. The CI/CD approach requires new tooling and a new mindset on the part of your team, and if the process isn’t managed well you can easily end up releasing broken software.

Here are 7 hard-learned lessons from our engineering team for making CI/CD a success.

Release in small increments. At Rainforest, we often release changes a few lines at a time. Rather than roll bug fixes and small features into a larger update, we push each individually into production as soon as it’s ready. These are still tested before release, but having the smaller chunks means that if something doesn’t work, it’s much easier to see what broke, and rolling back is trivial, fast and low risk.

Use pull requests. Pull requests allow you to separate the features you’re working on, update them, and then merge them back into the releasable branch. Github is your friend here. Our code always starts out on a feature branch -- committing directly to the master branch leaves too much room for error. When the author is convinced the code is ready to ship, they make a pull request. When unit tests have been run and the reviewer is satisfied, the code is merged to the develop branch. An automatic release from develop through to production happens from here out, covering QA, unit tests, migrations and releasing.

Find the right reviewer. Code reviews, while awesome, can turn into a major bottleneck if they're “hierarchical,” meaning each review has to come from a more senior developer. At Rainforest, our code review is almost entirely peer-based to keep things moving fast. If code requires additional scrutiny -- if it’s security-related, for instance -- we flag it for inspection automatically by at least two developers, including the CTO.

Benchmark for performance. While most checks involve quality and bug prevention, if performance is critical for your app, add an automated benchmark suite to your CI build. This is a great way to avoid inadvertently introducing performance regressions, as well as to make sure you hit your overall production performance targets.

Continuous QA. If you’re doing continuous delivery, you need continuous testing. Otherwise you’re constantly at risk of breaking things, or waiting for your QA team to catch up with your developers. Whatever you choose, be it automation or super-fast crowd-sourced testing, make sure it fits with whatever you think is reasonable for your release process. Most companies aim for 30 minutes end-to-end.

Automate wherever possible. This should go without saying, but automate as much of the process as you can to keep things moving quickly. This stops things getting dropped and improves consistency. We’ve automated processes for code review requests, unit testing, benchmarking, and, where possible, QA. Manual testing, unless it’s extremely cursory, can’t keep up with CI/CD. You must automate, or find some other method of comparable speed.

Build the right stack. Pick tools that integrate well with each other and fit into your team’s workflow. We use Github for hosting our code, CircleCI for CI/CD, and Heroku for our infrastructure. We also use Circlemator, an open source tool we released ourselves, and Fourchette, which has now been implemented in Heroku.

Russell Smith, Rainforest
Russell Smith, Rainforest

Clearly, CI/CD isn’t a magic bullet, but done right it can improve the quality of your software. Releasing code in small chunks is the least risky way to deploy, since fewer changes means there’s less chance of unforeseen bugs -- and easier debugging when bugs do appear. But it’s a big shift for dev teams, even compared to agile. I hope these tips will help to make it a success at your organization.

Russell Smith is the CTO & co-founder of Rainforest QA, the industry's only AI-powered crowdtesting platform that helps agile and continuous delivery engineering teams move faster. His specialties include development, developer workflow, DevOps, Linux, Debian, CI, benchmarking, profiling, bug fixing, performance, scalability, ops planning, capacity planning and modeling.

The InformationWeek community brings together IT practitioners and industry experts with IT advice, education, and opinions. We strive to highlight technology executives and subject matter experts and use their knowledge and experiences to help our audience of IT ... View Full Bio
We welcome your comments on this topic on our social media channels, or [contact us directly] with questions about the site.
Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
KentG555
50%
50%
KentG555,
User Rank: Apprentice
4/20/2017 | 12:34:50 PM
Misuse of Terms
Great article and great ideas,  however, software testing is really quality control, not quality assurance.
Slideshows
Top-Paying U.S. Cities for Data Scientists and Data Analysts
Cynthia Harvey, Freelance Journalist, InformationWeek,  11/5/2019
Slideshows
10 Strategic Technology Trends for 2020
Jessica Davis, Senior Editor, Enterprise Apps,  11/1/2019
Commentary
Is the Computer Science Degree Dead?
Guest Commentary, Guest Commentary,  11/6/2019
White Papers
Register for InformationWeek Newsletters
State of the Cloud
State of the Cloud
Cloud has drastically changed how IT organizations consume and deploy services in the digital age. This research report will delve into public, private and hybrid cloud adoption trends, with a special focus on infrastructure as a service and its role in the enterprise. Find out the challenges organizations are experiencing, and the technologies and strategies they are using to manage and mitigate those challenges today.
Video
Current Issue
Getting Started With Emerging Technologies
Looking to help your enterprise IT team ease the stress of putting new/emerging technologies such as AI, machine learning and IoT to work for their organizations? There are a few ways to get off on the right foot. In this report we share some expert advice on how to approach some of these seemingly daunting tech challenges.
Slideshows
Flash Poll