CEO Guide to Optimizing Technology Organizational Structure

The What and Why of Technology Organizational Structure

Technology and Product organizational structures commonly oscillate between:

• Centralized and decentralized (single monolithic organization serving all business functions or separate teams reporting to separate business functions)
• Functional (similar skillsets in the same team e.g. all software folks together) or cross functional (all skillsets necessary to complete a delivery in the same team such as software and hardware being together) or matrixed (non-durable groups of people working together, each reporting to separate managers).

In recent years Technology Organizational Structures have trended towards smaller, autonomous, cross-functional, business aligned product engineering teams which are ideal for multiple reasons subsequently described.

Although this approach may sound straight forward to put in place there are multiple pitfalls and nuances to optimizing the outcomes of durable, business aligned, product engineering teams to consider as you evolve your org structure.

Let’s start by discussing why Technology Org Structure is so important to business outcomes:

• Product development needs to move as quickly as possible to thrive in an environment of accelerating competition, technical advances, and rapid business change. Optimizing the organization for Time to Market (TTM) is therefore table stakes. Even if you produce a product that is only periodically released due to the customers need/desire to pace their integration of new functionality, it is still desirable to maximize the output of your product engineering organization to get the most output for each dollar spent and complete new functionality as quickly as possible.
• An optimal Technology Org Structure also helps improve quality at an efficient cost and can be an orthogonal outcome if the product architecture is also aligned to result in autonomous teams that can test and release independently and safely (ie. with minimal or no risk to impacting other parts of the product).
• A third key benefit of optimizing the Technology Org Structure towards autonomous, durable and empowered teams is to enable mastery which improves morale and motivation. This creates a flywheel of positive outcomes in terms of employee loyalty, dedication, productivity, and employment brand in attracting new talent.

So how do you get the org structure right when striving to improve TTM, drive innovation throughout the organization and recruit and retain the best and brightest? There are a few key dimensions to address.

Team Size

One of the simplest summaries of optimal team size was coined by Jeff Bezos.

In practice we find that teams of 8-12 people are an ideal size for communication, collaboration, team dynamics overall and therefore output.

As team members are added, there is more time spent in communicating and less time working, to the point that the team may actually spend much more time communicating and much less time working on ‘real output’. The equation below reminds us that communication overhead increases exponentially as the team size grows.

Architectural Alignment to Autonomous Teams

Product architecture also plays a very important role in actually enabling optimally sized teams to operate autonomously. Often, we see clients with teams that are appropriately sized and are aligned to a portion of the product that they can have primary ownership of, BUT if the application is architected so that there are still many dependencies across product areas then much of the value of teams aligned to a part of the product is lost. In this case teams still have to coordinate heavily with other teams to make changes safely in the application because 'their part' of the product is so heavily intertwined with other teams' parts of the product at the code level. This situation is most common in older monolithic code bases that have not been decomposed into separately deployable services. However, this pitfall of dependencies can also exist in a more modern distributed, services-based architecture if services are not well constructed and/or do not follow best practice patterns of interaction and graceful degradation in case of failure. The volume of communication happening externally for a team can be measured in various product engineering tools (e.g. Slack) but often a simple survey or discussion with team members can help uncover if dependencies are still an issue hindering team autonomy and therefore TTM.

Given product architecture is key to fully achieving the benefits of small teams, it’s also important to consider how many independently deployable services (/applications) a team is responsible for. Ideally, a single team is responsible for a single service. As distributed architectures have become more popular, we often see a team owning multiple services. This is not ideal as it results in unnecessary complexity (which slows execution and increases operational risk). Essentially, many companies have taken the idea of microservices too far beyond what their business currently needs and in doing so introduced suboptimal complexity. If a team owns 2-3 services, it may be suboptimal but manageable, as the team may divide along service boundaries. However, a team should NEVER share ownership of a service with another team.

A good reminder of the results of aligning teams to product areas comes from Conway's law, dating all the way back to 1968. His research, popularized later by Fred Brooks, reminds us that architecture will mimic the communication structures of the organization which heavily depend on the underlying broader organizational structure. However, as described above, if there is a 'pre-existing architecture' that entangles teams across the product domains each team is aligned to, this also must be addressed in order for the teams to be autonomous (by breaking dependencies across individual product aligned teams).

Team Construction

Equally important to small team size is the construction of skills and roles in a team. Initially Agile Scrum teams were composed of a Product Manager (PM), Engineers, QA (quality assurance) and a scrum master.

Now the complement of skills is often expanded to Architecture, DevOps, Site Reliability Engineering (SRE), user experience design and periodically data engineering, data science and security.

Bringing DevOps and SRE skills into the individual scrum teams enables the team to have all the skills needed to deploy, monitor and maintain their application in production. These skills are the most common addition to Product, Engineering and Quality, however, the other aforementioned roles are appropriate to dedicate to scrum teams in many cases based on the requirements of the product and industry the product operates in. For example, in high security government, financial or health care applications, a higher level of security may be required and therefore appropriate to embed into some or all individual scrum teams.

For design resources, it’s common to have a designer span 2 scrum teams but still maintain alignment to specific parts of the product rather than being assigned to work in more of a project mode where they work on all aspects of a larger product(s).

Quality also deserves further discussion as many organizations are striving to move away from having a separate role dedicated to testing. Instead, quality is automated as part of the engineering process by following disciplined Test Driven Development (TDD) practices that ensure automated tests are built before the actual code is written and all tests are automated. Regardless of whether your company is on this path, it is optimal to have the objective to automate as many tests as possible and to follow quality engineering best practices rather than having dedicated manual testers who, by definition, test after the code is fully written.

The key benefit to small autonomous teams is of course, once again – TTM. When a team has all the skills necessary to successfully design, build and deploy new code to production, speed will be optimized. If you are unsure if you need more specialized roles like, having a conversation about things that are slowing a team down will often unearth skills that should be partially or wholly dedicated to scrum teams. The most common areas that slow teams down if they are not embedded are DevOps/SRE and in many industries, security.

Team Diversity

Diversity is another area that is gaining focus and priority for executives.

Diversity in teams is of course just good business but is also critical for driving innovation as it enables cognitive (good) conflict in exploring options and approaches. Cognitive conflict is all about exploring options regarding the “what and how” whereas affective (bad) conflict focuses on “the who” and does not add value in coming to an optimal product idea and implementation.

Cognitive discussion with a small diverse team results in more possibilities being explored as opposed to the ‘echo chamber’ of ideation within a team of similar skills and backgrounds.

Having cross-functional roles in a team is a key aspect of this diversity of thought, networks, and past experiences. That said, hiring for diversity in age, gender, ethnicity (and many more dimensions of ‘humanity’) are all critical to maximizing diversity to enable creative outcomes.

It's important to also understand that experiential diversity initially also results in affective (bad) conflict until the team is through the initial phases of Tuckman's cycle, namely Forming and Storming, and has built trust and understanding and is aligned to a common objective. The diagram below depicts dimensions of diversity that positively or negatively impact a teams' ability to collaborate and innovate effectively. Since experiential diversity leads to affective conflict, leadership must monitor for this and help the team manage through this phase and may need to help 'stamp out affective conflict' if the phases of team bonding do not do this naturally. Once any initial affective conflict is managed out, then experiential diversity will positively contribute to cognitive (good) conflict and innovation.

Team Durability, Alignment and Reporting Structure

Having small, cross-functional and autonomous teams is a great start, but where should they report from there? Ideally these individual teams should be aligned to a part of the product they can own for a period of at least 2-3 years and not less than 6 months. We want the team to be together at least 6 months to have a chance to reach Tuckman’s “performing” phase and we can help. Two to 3 years is an ideal stage to maximize this performance while minimizing the negative effects of overly long tenured teams and “group think” that starts to lower team innovation.

In a scaled business, many scrum teams (potentially dozens) align directly to a line of business. In this case the primary reporting relationship should be to the CEO of that line of business. Again, this supports the idea that the CEO of a line of business has everything they need to be successful in optimizing business outcomes.

A regular refrain in having multiple different product engineering organizations is then duplication of effort in tools, processes and norms across the full org. In addition, there can be concern about enterprise architecture, security posture as well as overall continuous improvement across disciplines (Engineering, Product, Quality etc.). This can be solved by having smaller centralized functional leadership teams which individual engineering teams have a dotted line relationship to.

All organizational structures involve trade-offs so it is critical to be clear on what you are optimizing for. It is imperative to optimize for speed to market to remain competitive and therefore the trade-off between primary line of business alignment versus ‘functional mastery’ is a relatively easy one to make. That said, it’s important to be clear on what the purpose and purview of centralized functional leadership is. The main objectives of secondary dotted-line functional alignment should be to strike a balance of cross organizational efficiency (alignment of tools, processes, hiring practices etc.), continuous improvement and skills development while not compromising speed to market in each product area or line of business.

Key points for CEOs and Board members

1. Product engineering teams should be small and cross functional. Most larger organizations do periodic review of their structures as there is natural drift from optimal, but ideally this information is transparent in the People/HR systems such that it's easy to spot potential problem areas and address them.
2. The benefits of speed to market will only be realized if the architecture is distributed and follows good patterns for interactions between services. Otherwise, smaller teams will still require an exponential amount of communication outside their team in order to make changes safely.
3. Product engineering teams’ primary solid line reporting should be into the line of Business that owns the product area they work on. Their secondary line of reporting should be to their functional role in order to balance efficiency and support mastery in each role.
4. Autonomous teams focused on a part of the product should be kept together for as long as possible, while of course still considering the need to cross-pollinate, promote, etc.
5. Senior leadership should monitor for the signs of affective (role-based) conflict on an ongoing basis and take action to eliminate this value destroying conflict by addressing the root cause(s), e.g. roles that are not dedicated such as DevOps or Security Engineering, poor individual contributor or manager behaviors etc.
6. The CTO should report directly into the CEO in most cases. Even in hard goods industries, the digital experience has become so paramount that there are very few cases where the CTO should not have a seat at the executive table. Similarly, if there is a Chief Product Officer (CPO), they should also report into the CEO (in some organizations, the CTO/CPO is the same person which also works effectively but ideally engineering is not buried under a COO with many responsibilities).

Common CEO questions about Technology Org Structure

My teams are functionally organized right now – how do I move towards this structure?

Some organizations have started with initially aligning engineering to scrum teams and lines of business and later aligning smaller functions such as Product or Quality engineering. While this is a start, we recommend having a consistent structure across all functions and therefore moving all cross-functional roles into their autonomous business aligned teams as their primary reporting relationship and dotted line to their function.

You may also start by bootstrapping with a few teams but this can also create more confusion. Therefore, ideally the structure change is made consistently across functions and product lines.

Should I make this change if I have a monolithic code base that I’m trying to decompose for speed to market? Or after we have re-architected?

Yes, you should make this change even if you have a monolithic code base. However, the decomposition of a monolith into separately deployable smaller services aligned to vertical slices of the products’ functionality will require time and attention at the CTO level in order to ensure progress and alignment to a cohesive outcome. As previously mentioned, distributed architectures can also result in complex cross team dependencies if the new architecture doesn’t follow best practices in service-to-service communication, handling of failure of other services, etc.

What Technology Org Structure is not

Like many other dimensions of good product development, organizational structure is only one part of the equation. However, it IS an underpinning to achieving optimal speed to market which will not be realized without having this foundation in place.