The Agile Organization
We’ve done a lot of work with organizations attempting to become more Agile by implementing Agile development practices. One common problem we see time and time again is that the “old school” way of defining organization structure starts to lose its value in an Agile world. Here I am specifically talking about organization structures developed around functional roles such as development (or engineering), QA, Operations, Infrastructure, etc.
This old method of organizing, which resembles a Y axis split within the AKF scale cube served our industry well for a long time. And, for many organizations, it can continue to work well. It particularly works well in organizations that follow waterfall models as the organization structure mimics the flow and passage of work through certain gates. The structure is also comforting in its familiarity as most long tenured managers and individual contributors have worked within similarly structured organizations their entire professional careers.
But in the Agile world, this organization structure doesn’t add as much value as in the Waterfall world. In fact, I argue that it’s counter-productive in many ways. The first and perhaps most benign issue is that the actual structure of the organization does not foster work-flow. Unlike waterfall development where one group hands off a project to another in phases (development to QA, QA to operations, etc), Agile methods seek to develop and deploy seamlessly. To be successful the Agile team needs representation from multiple stakeholders within functional groups. As individuals now spend most of their time in cross functional teams, what value does the functional group offer? In essence, these functional organizations become the analog to the “home room” in school.
The next problem is the inherent conflict created between the Agile team and the functional organization. To be truly effective, the team must be empowered to some degree. What power or responsibility does the functional leader then have? If he or she isn’t responsible for a specific product, are they to be given some sort of veto power? Such a notion has meaning in the waterfall world but really runs counter to the time to market and discovery objectives of Agile methods. The resulting affective conflict simply doesn’t add value to the overall product. In fact, as research shows, it destroys value. Some proponents of continuing with functional organizations might indicate that the functional groups allow for more effective management and mentoring of individuals within their domain. Given how little time managers truly spend on mentoring relative to other tasks, I highly doubt this is the case for most organizations. Our experience is that the functional groups spend more time arguing over ownership of certain decisions (affective conflict) rather than mentoring, training and evaluating individual contributors.
Perhaps the largest problem – larger than the lack of support for work flow and the creation of conflict – is that implementing agile processes across functional organizations sub-optimizes innovation. Research indicates that innovation happens most frequently and beneficially within groups of individuals with diverse and non-overlapping experience across a number of domains (functional and experiential diversity) and with non-redundant links to individuals outside of their organization. By engaging in beneficial debate (cognitive conflict) on approaches to a certain problem or opportunity, the perspective and networks brought by each person widens the potential solution set. Alas, this is the true unheralded value of agile development teams that properly incorporate multiple disciplines within the team (QA, dev, product, infrastructure). Each of these individuals not only brings new and valuable expertise in how to develop a product, they also have contacts outside the organization unlikely to be matched by each of their peers. Innovation quality and frequency therefore increases. But the inherent conflict in multiple competing organizational affiliations will dampen this innovation. So not only is there conflict and a lack of workflow, the potential major benefit is removed or at the very least diminished.
Having discussed the problems inherent to functional organizations and agile processes, we’ll next discuss how a company might “organize” around agile to be more effective.