CEO Guide to DevOps

What is DevOps?

DevOps is a cultural philosophy and set of practices that breaks down functional silos between software Development (Dev) and Operations (Ops). Historically these functions were separated with different skill sets and tools and, in some cases, objectives.

Patrick Debois coined the term DevOps first in 2009. Debois was a pioneer in defining the associated practices and cultural shift to bring Development and Operations together to improve the delivery of software to production.

Before the dawn of DevOps, the functions of Development on the left of the virtuous cycle below were seen more as a linear set of steps with a handoff 'over the wall' between Development and Operations teams. Development teams were primarily focused on building software while Operations teams deployed the code to the production environment and operated the product including tasks like adding capacity, performing maintenance upgrades, monitoring, as well as detecting and remediating issues in production.

As technology has advanced, the ability to ‘automate everything’ has become more accessible with ever-accelerating innovation in the Continuous Integration and Continuous Deployment (CI/CD) tools and frameworks space. The existence of an end-to-end toolset that facilitates seamless flow of code changes has accelerated the breakdown of silos between building and operating a platform and enabled a smoother, more efficient, and higher quality path from completing a code change to making it available to customers in production.

Although DevOps spans into other capabilities in this cycle it is primarily concerned with automating the path of steps from a code change being committed by a Developer, to safely deploying that code change to production.

DevOps done well results in faster time to market and higher quality and is complimentary to other Agile development practices. As previously stated, it is primarily focused on the development pipeline and reducing the time and effort required to move any change from a Developer's environment to production. Ideally, all aspects of testing, security scanning and integration into the main code line are automated, to achieve the outcomes of the fastest possible time to market, while maintaining high quality through automated testing.

The vision of DevOps has been made possible to a large degree based on the constantly accelerating pace of technology change, including:

• Agile development practices that break down silos by creating cross-functional Scrum teams that have everything they need to deliver software within the team itself (e.g. Dev, QA, Product, DevOps etc.)
• Increasingly easier to use automation tools and capabilities that make it possible to automate all aspects in the process of moving a software change from the Developer’s environment through quality and security checks to production deployment
• The evolution of public cloud computing which has accelerated the breadth and usability of automation tools to make DevOps capabilities more accessible to all companies trying to improve their time to market with quality (the level of expertise needed to achieve the DevOps vision is constantly being lowered/improved based on the tools available).

The evolution of DevOps has also positively impacted the dynamics of collaboration between Development and Operations. Engineering was historically focused on getting as many features shipped to customers as possible, while Operations was focused on keeping the software product running smoothly. This, rather obviously, results in a level of tension and affective (bad) conflict between the two teams that if left unchecked, significantly impeded the Business’s forward progress. Engineering was motivated to ship code; Operations was motivated to reduce the amount of change, manage risk and keep everything running without issues. In addition to this, many companies also did not align objectives across the two teams, making the problem worse as Development's objectives were focused on getting product out the door and Operations were focused on keeping the product running and stable (in direct conflict).

In addition, before the rise in popularity of Agile development practices, Development and Operations were commonly in different functional organizations, reporting to different leaders, rather than collaborating as part of cross-functional engineering (Scrum/agile) team. This exacerbated the level of affective conflict which is centered more around “the who” rather than focusing on common business outcomes (TTM, customer quality etc.) in an aligned and collaborative way.

The rise of DevOps has truly started to break down the walls between Dev and Ops. DevOps practices, coupled with the proper cross-functional organizational structure at the scrum team level, as well as shared objectives focused on common customer outcomes, is paving the way for Dev and Ops to work much more harmoniously together.

Why is DevOps important?

DevOps is important because it is a critical practice to achieving one, if not the most important, business objective – speed to market! (Equally important is building the "right features,” which customers will actually use/buy).

In addition to time to market, DevOps also supports better quality as all quality checks are automated and enforced in the pipeline. Therefore, testing is consistent, and changes can be rejected when automated testing fails. The level of automation that can be achieved with DevOps practices significantly reduces the likelihood of human error.

Lastly, as previously mentioned, DevOps done well breaks down the silos between the functional expertise of Developers and Operations. Not only does this improve speed to market and quality, it also improves morale and ongoing collaboration between the teams as they evolve to understand each other’s perspectives and appreciate the skillsets that each brings.

Key points for CEOs and Board members

1. DevOps is just as much a culture of cross-team collaboration, automation and quality, as it is a skillset or a role.
2. In order to scale and achieve the optimal collaboration model, DevOps capabilities MUST be embedded in the Agile (Scrum, squad, pod) teams. The direct, solid line alignment of DevOps team members should be to the Agile squad first and to any DevOps central function second (dotted line).
3. Since DevOps is a specialized skillset, just like Database Administration, Quality Engineering or Security, scaled organizations should usually have a small DevOps Enabling team that works with all engineering teams to drive best practices, set standard practices and tools as appropriate based on the size and culture of the organization. The members of this central team can be rotated into Scrum teams so that they are not an Ivory Tower function, and they truly understand the challenges of the teams they support.
4. When first starting to build a DevOps capability it is ideal to bootstrap and accelerate progress by hiring a small number of external hands-on experts while simultaneously intensely training internal team members. Internal team members can come from either/both Development and Operations. Most importantly internal candidates should be avid learners, have some level of coding/scripting capabilities and be highly collaborative.
5. With regards to goal setting and alignment, it’s important for Agile teams to have goals aligned to customer outcomes. This practice should eliminate conflicting goals. Any notion of a centralized DevOps enabling function should also have customer outcome goals to ensure alignment and success.

Common CEO questions about DevOps

How do I know if my Engineering teams’ DevOps capabilities are where they need to be?

Based on DevOps purpose to optimize the flow of code from development to production, some of the key metrics to consider are time to market and quality related. You can start by understanding the Deployment Frequency for changes on an ongoing basis.

The possibilities for quality metrics are endless, but most critical is customer quality which is indicated by Change Failure Rate (changes that fail in production and are rolled back) and number and severity of customer impacting incidents.

How should DevOps be organized?

As previously outlined, it’s most important that DevOps capabilities are embedded directly into engineering Scrum teams. However, in many organizations, DevOps may start with just a few experts who support more than one Scrum team. If there is a central DevOps function it should be in combination with DevOps embedded directly in teams (and include rotation from the central function to individual teams).

What DevOps is not

It’s NOT SRE! (Site Reliability Engineering)

DevOps is a philosophy and relatively broad set of practices that goes ‘hand in glove’ with many aspects of Site Reliability Engineering but there are differences as well. DevOps is generally more focused on optimizing the introduction of change into production (the development ‘pipeline’), while SRE is more focused on optimizing the operation of code in production, in terms of scale, reliability and resiliency.

The DevOp/SRE contrast is also succinctly stated by harness.io.

“In a nutshell, DevOps Engineers are ops-focused engineers who solve development pipeline problems. Site Reliability Engineers are development-focused engineers who solve operational/scale/reliability problems.”

This is a good simple contrast, although often DevOps and SRE skillsets have significant overlap in terms of tooling, scripting/coding capabilities, and the passion for efficiency and quality in the end-to-end process (customer quality!).