Visit 100 websites and 100 companies and you’ll find neither a consistent definition nor a consistent implementation for Product Operations. As Marty Cagan points out in his interactions with Product teams he has seen at least six different approaches. We highly recommend a complete read of Marty’s post as it highlights the different models he’s seen, and the issues with five of those models.
Conflicting definitions and implementations aside, there is a very real need for the capabilities defined within this article for product organizations operating “at scale”. Scale here refers to both the size and throughput of the total engineering and product organizations. Most of the below capabilities are focused on consistency of outcomes, the ability to improve upon past successes and avoid past mistakes within companies with more than 100 total engineers and product managers (or 8 to 13 cross functional teams). Many of these capabilities are missing in most of the companies with which we work, or when the capabilities exist, they lack repeatability and consistency across teams.
Key Capabilities and Outcomes of Product Operations
For the purposes of this post, we will refer to the below as the AKF view on Product Operations. The capabilities are as follows:
- Standardization and centralization of metrics, market, and product insights.
Absent a centralized and consistent approach to metrics, each team develops its own path to perform qualitative and quantitative analysis of market opportunities and competing solutions. This in turn makes comparing capabilities at an executive level difficult as skew in approaches lead to inconsistencies in analysis. The reader is left to interpret both the analysis and the methodologies employed, increasing the cost of executive evaluation of opportunities, and increasing the probability of errors resulting from inconsistencies.
Further, not every product team will have equivalent skills (both qualitative and quantitative) in performing analysis and some may even lack the minimum skills to select the correct approach for analysis. When, for instance, is structural equation modeling the appropriate tool over linear or curvilinear regression? When should exploratory factor analysis be used in evaluating survey responses for psychometric data? These skills are not always present in each product team, and the cost to hire them for every product team may be both dilutive and lead to underutilized high value assets.
- Standardization and governance of best practices
As indicated above, absent appropriate oversight, the processes employed by teams often drift. This drift represents an increased cost to the company as each team spends time defining and improving processes their own processes thereby eliminating the leverage one would expect as a company scales. In addition to the lost opportunity of leverage, there also exists a high probability that the skew in process will result in differing outcomes and therefore varied quality of outcomes. The processes in question range from product specific tasks such as interviews and planning to holistic cross-team processes such as sprint planning and reporting. Documentations and templates used across the organization also fall into this category. Below is a (partial) list of recommendations for the product operations teams oversight and management:
- Release processes
- Customer onboarding processes
- Development of standard templates and reference material
- Interview guidelines and processes
- Standardization of Tools
Just as DevOps is often responsible for the implementation and standardization of tools jointly used by engineering and operations, so should the Product Ops team be responsible for at the very least ensuring consistency in the tooling primarily used by product managers. Everything from tools addressing customer product utilization to workflow processes should be standardized across teams to ensure the company gets leverage with scale and consistency/repeatability in outcomes and quality between teams.
How NOT to Implement Product Operations
Centralized shared services teams are problematic for a company that wants to move quickly for several reasons. They often become bureaucratic entities reminiscent of either a bad DMV experience or a horrible flight experience with an ill-tempered flight attendant. Its common for a centralized shared services group to become distant from what really creates customer value and in so doing form an identity around their organizational power thereby engendering affective or role-based conflict. Further, as many centralized teams don’t often “consume” the solutions they produce they often lack the incentives to learn from the deficiencies of their solutions and improve upon them; the lack of market forces in any monopoly lead to bad outcomes and poor customer experiences.
How to Implement Product Operations
To combat the issues above, we think it’s critically important to ensure that members of product operations teams be “embedded”, even if temporarily, into teams that do “real work” (we make the same recommendations for DevOps teams).
- Analysts or data scientists should be assigned to various teams and work alongside those teams when requested. Not every team needs a full-time person, but the person with whom they work should be the same person every time one is needed.
- People focused on tooling should be a rotational position comprised of engineers and product managers. Such rotations will ensure that the people doing real customer facing work and using the tools are also helping to develop and improve upon them. This eliminates the moral hazard of one team dictating the way the other works without experiencing the implications of such dictation. Further, the theory gets informed by the practice thereby implementing a self-healing feedback loop.
- People focused on best practice development should similarly come from the ranks of people doing “real work”, and such people should periodically return to do “real work”. Again, a rotational system will work well here.