Some DevOps teams are well-oiled machines, but it's taken time to get there. Perhaps they didn't have the right culture or processes in place to start. Maybe they lacked some tooling that could help facilitate more efficient processes.
"There isn't one way to do DevOps," said Jared Murrell, director of DevOps, engineering & communities of practice at Github. "It's not simple to quickly and perfectly 'do DevOps.' DevOps is defined by the people building software and how they work together. Naturally, there are some hurdles to overcome."
DevOps team members need to understand something about what the others do so they can all work together as a cohesive cross-functional team. That doesn't just happen because later stage functions have been responsible for identifying flaws in code, which is a different mindset than working as a team to continuously improve the speed at which code is delivered, it's quality and internal processes.
"You won't have successful DevOps unless the whole team embraces responsibility and contributes to the larger goal," said Murrell. "Instilling this responsibility allows teams to move forward together and makes each team member feel they have a stake in the overall success of the team."
DevOps began with the idea of getting Dev and Ops working closer together, but the concept doesn't end there as evidenced by the emergence of modified terms such as "DevTestOps" and "DevSecOps." However, DevOps teams also need to work with other functions such as cybersecurity and governance.
"For DevOps work, you need strong communication and integration between teams so a DevOps team can take ownership of their own governance and risk [without] putting the organization at risk or running into a brick wall from another function in the company," said James Bores, principal at Bores Security Consultancy.
These days, DevOps teams also need to concern themselves with data. Brian Platz, co-CEO and cofounder of blockchain-based graph database platform Fluree, said DevOps can contribute to an enterprise data bottleneck if it lacks the proper data infrastructure.
"DevOps will need to create and likewise demand access to high quality, rich and real-time data, but if data rules and standards aren't created and enforced as a strategic priority, the acceleration of products and services under DevOps will simultaneously accelerate data silos and leave behind a wasteland of duplicated and partial information," said Platz. "DevOps decision makers must collaborate with data professionals in their organization to define the key data stakeholders and derived data standards to integrate into their toolset and process."
DevOps requires automation to meet its goal of faster value delivery. As DevOps teams move toward CI/CD, more of the pipeline must become automated.
"Automation is a huge part of successful DevOps, but teams need to determine what is beneficial to automate, and where to start," said Github's Murrell.
Denis Leclair, VP of engineering at sponsored ad campaign creation and management platform provider Trellis, said in his experience, the two most frequently cited challenges in deploying DevOps in organizations include the upfront investments needed in tools and analytics to enable the successful automation of workflows across the range of value streams and reskilling the workforce to be effective in the new model. One example is software quality assurance (SQA).
"For the benefits of DevOps to be recognized as scale, testing and feedback must be automated," said Leclair. "Developers need to embrace a testing mindset (e.g., TDD). On the other hand, SQA engineers need to become proficient in designing and developing large [and] complex test automation code systems -- that is to say, they need to become more skilled as software developers."
DevOps teams tend to morph into DevSecOps teams because DevOps and a separate security function isn't working well enough. DevOps teams are under constant pressure to reduce application development and delivery cycles, which can and often does impact product quality including security. Shifting security testing left helps, but it doesn't produce the same results on its own as including security expertise on the team so security is top of mind for everyone.
"Developers understand how to write code. What they're missing is the perspective [of] what attackers do and how they do it," said Kevin Breen, director of cyber threat research at cyber security skills platform provider Immersive Labs.
Breen advocates security champions who understand software development and cybersecurity because they can explain cybersecurity within the context of the code the developer is building. In addition, teams should have automated application vulnerability scanners that are part of the pipeline, although QA should also be taught how to use some security testing tools so they can run SQL injection tests, for example.
"It's about empowering developers, QA and operations to use the right tools at the right place and not be afraid to go to the security team," said Breen.
Roey Eliyahu, CEO and co-founder of Salt Security, said his customers achieve the most success when they prioritize both pre-production and post-production security.
"Despite customers' best efforts, applications frequently roll out with bugs and vulnerabilities," said Eliyahu. "Modern mobile and web applications that rely on multiple APIs create a complex web of logic and are especially susceptible to such challenges. With API-based applications, many gaps do not surface until they're in production."
Managing cloud spend isn't just a DevOps issue, but there are some nasty cost-related surprises that DevOps teams can run into according to Jeff Valentine, CTO at cloud management platform CloudCheckr.
One is relying on AWS Pricing Calculator results and presenting them to the business as-is when the tool can't factor in what the business is trying to achieve, how the team will run the software or what mistakes the DevOps team may make. Another "gotcha" is failing to consider the cost of bandwidth, which can account for 5% or 10% of a bill. A third pitfall is believing that shutting off an EC2 instance will stop costs from accruing when the EBS volume was never deallocated.
"There are all these weird analogues that were never 'a thing' on premise so they didn't realize they had to ask about it," said Valentine. "Also, they tend not taking advantage of cost savings plans because they just don't want to spend the time learning."
DevOps is a journey that requires patience and dedication on everyone's part. While strong leadership is needed to succeed, there are technology, process and people elements, all of which need to be addressed.
DevOps is a mindset and a practice that focuses on continuous improvement on several levels including building and releasing better quality code faster, improving intra and inter-team communication and collaboration, and overcoming other common barriers such as automation and using cloud spend wisely.
For more on DevOps, follow up with these stories: