Agile Is King, But Continuous Integration Is An Elusive Goal

More on DevOps
Live at Interop ITX

When it comes to enterprise software development, agile is in, and continuous integration is deemed a worthy goal. Many organizations report that they have adopted some form of agile development. Continuous integration, on the other hand, where frequent application updates are meshed with production systems, remains a goal that's growing in popularity but implemented by only a few.

Those were some of the key responses of of 732 software professionals who participated in a recent survey, Testing Trends In 2017, conducted by the survey firm Dimension Data. The online survey was called to the attention of technology professionals listed in a database responsible for Web and model application testing. The survey was sponsored by Sauce Labs, supplier of a cloud-based, automated testing platform.

Thirty-five percent of the companies participating had 1,000 employees or more; 33% had 100-1,000 employees. The remaining 32% were small companies of under 100 employees. Eighty-seven percent of the participants were working in North America or Europe.

The number reporting that all their software teams were engaged in agile development was 32% but a total of 94% said many or some teams were committed to using the agile process. Only 6% said agile was either not used or still in the pilot project phase, an indicator of how thoroughly the agile development process has penetrated IT staffs.

Want to learn more about how to move your organization into DevOps? See 8 Steps To A Successful DevOps Transition.

Continuous integration with an ability to deploy hourly, often described as an end goal of adopting an agile development process, was cited by 28% as the destination they were shooting for. However, only 14% were actually doing so. Hourly continuous integration a year ago was a goal for only 18%. The added 10% a year late shows how quickly continuous integration is rising in the consciousness of development staffs. It's rising faster than the actual ability to deploy, currently at 14%, but a year ago a similar Dimension Labs survey showed it to be 10%.

On the other hand, everyone is trying to practice the meshing of a software update into a production system. Thirty-five percent of respodents said they could integrate updates daily and 17% said weekly. Another 20% do updates on a "less than weekly basis" but still more frequently than the six-month or annual update periods practiced by development staffs of yore. The less frequent periods are actually sinking in popularly slowly as the hourly update category grew by 10%, with an obvious shift in sentiment toward rapid updates.

Another revealing result was the frequency with which bugs are fixed. When bugs aren't affecting production runs, they can linger on the development staff's list of things to do. That's still holding true, though improving slightly. The number that said they fixed bugs immediately remained flat from 2016 to 2017, at 23%. The number that said "within a few working days" improved from 48% last year to 51% this year. Those reporting bug fixes took "more than a few days" went from 5% last year to 4% this year.

The 2017 report said: "This is clearly an area where more work can be done by testing and development teams, since the benefits of fixing bugs immediately are well established."

One way to reduce the number of bugs phased into production is by writing tests for new code when it's checked into the code management and build system. Nineteen percent reported they do so, leaving 81% who do not. Fifty-four percent said they leave it to the later phase of quality assurance and testing. Another 22% said tests weren't written until after development was completed, a practice that is more associated with the old waterfall method of software development than agile methology. Five percent said they didn't write tests.

The survey showed another trend in that test automation is increasing. Eighty-seven percent of development teams have adopted some form of test automation, but the use of tools varied widely among survey participants. Open source tools, such as Selenium and Appium, were popular with 70% of respondents; proprietary tools were used by 12% and manual testing was conducted by 13%. Fifty-two percent said they were using some form of an automated test framework. At the same time, 42% said their testing process set up and execution were "mostly" or "entirely" manual.

The use of the public cloud for testing has increased 9 percentage points in the last two years, registering at 43% in 2015, 45% in 2016 and 52% in the 2017 report. Each report of course refers to practices conducted the year before.

DevOps is more than a buzz word in 2017 with 88% of respondents saying they have adopted its practices, are in the process of adopting it or are considering it. Six percent have no plans to turn to a DevOps approach. The definition used in this surveys was: "a practice that emphasizes the collaboration and communication of both software developers and other IT professionals while automating the processes of software delivery…" In the vernacular, DevOps often is taken to mean the collaboration between developers and IT operations managers, with a shared responsibility for how the software runs in production.

Although interest is high, only 10% reported that they have fully adopted DevOps across the company. The largest groups had a few teams using it, 24%, or have just started using it, 25%. Another 15% are still considering using the approach.

The most frequent participants in the survey were developers at 23%; 19% were test engineers; 17% were QA engineers and 9% were QA team managers. Another 8% were DevOps advocates or specialists. Six percent were architects and another 6% were development team managers.

The sponsor of the survey, Sauce Labs, was founded in 2008 in San Francisco and uses Selenium-based, cross-browser testing in its cloud service.