Every few years or so, the management consulting community latches onto a new cure for everything that ails large (i.e. potential customer) organisations. The hype cycle unfolds in predictable phases: the new practice seems to work in a number of organisations and gets highly publicised, companies of all sizes and kinds are encouraged to adopt it, a few prominent failures occur, and infatuation gives way to disillusionment as the magic medicine doesn’t seem so magic anymore. The cycle unfolds again (and again) as the fallen wonder gets replaced by the “next big thing”. In the process, a perfectly sensible practice that could work well for some organisations to solve some problems may become unfashionable enough to be rejected by most organisations. The baby is gone with the bathwater.
We fear that a similar fate awaits the latest panacea – the set of organisational practices that have come to be known as “Agile”. At this point, every top management consulting company in the world has some version of a “see how Agile you are” diagnostic survey, as well as a “here is how we can help you become Agile” offering advertised on its website. The first premise, often stated explicitly, is that every company, whether an FMCG giant or a national bank, needs to become Agile. Fair enough – who wants to be “clumsy”?
The second, not so explicit, is that the particular set of organisational practices being advertised, largely revolving around the use of cross-functional teams with self-organising elements, is the path to get there. The evidence for the second premise is simply non-existent at this point. In fact, everything we know from the study of organisation design suggests it is quite unlikely to be true.
To be clear, there is no disputing the appeal of organisations that are both efficient and innovative, which seems to be the promise of “Agile”. This is a problem that organisational scientists have been studying for a very long time under a variety of names (“ambidexterity”, “exploration vs. exploitation”, “organic vs. mechanistic structures” to name but a few). We think we understand quite well why it is hard to accomplish both, as well as the nuances around the kinds of solutions that are possible under specific conditions.
We’ll go further: We are fans of Agile practices as a solution to a distinct class of problems in certain contexts, such as those involving a search, be it for new designs, product features or even management policies. Agile practices use rapid, parallel and inexpensive iterations to solve search problems. If there are parts of your organisation that can benefit from this kind of problem solving, then Agile design may be for you. Even then, not all search problems may benefit from Agile practices. The relevance of Agile is murkier still if the context is one of execution, e.g. when the firm seeks to attain high levels of reliability with well understood processes.
What we want to caution against is the indiscriminate application of Agile beyond search situations, in large part because the inevitable disillusionment may then discourage even sensible adoption. Let’s keep that poor baby from disappearing with the bathwater.
Where Agile comes from
Agile practices grew out of a product development framework widely adopted by the software industry. Also called “scrum”, its central principle is to set up highly autonomous and independent teams, each working on a different aspect of a larger, complex problem. Instead of marching in bureaucratic formation, they are given freedom to “sprint”, i.e. rapidly advance in parallel towards a solution with minimal top-down oversight. Interdependencies between the various teams are managed through frequent meetings and any rework is accepted as part of the iterative process. Upon completion of the immediate task, teams either dissolve or take on a new project. When it works well, it yields a heady mix of autonomy, immediate and measurable accomplishment, as well as a sense of team cohesion.
The logic behind extending Agile to entire organisations is that they increasingly face conditions very similar to those of software organisations: the pressure to excel at rapid search and to be egalitarian, in a bid to keep knowledge workers motivated. Therefore, the argument goes, we should do what they (seem to) do successfully, which is Agile.
As we see it, there are three problems with this argument. First, the importance of good old-fashioned execution relative to search varies a lot across sectors. Not all are as search-intensive as software development. Second, we are not sure these practices work as effectively outside software development, even for search problems. Third, even in the software industry, the evidence that Agile practices can be scaled effectively is still thin.
Speeding through rapid parallel development is a chimera unless the pieces can truly be worked on in parallel. Production technologies vary enormously in this regard (a property known as “decomposability”). All that parallel sprinting can end in a crash at the finish line unless the pieces fit together. There is no obvious reason to trust that the degree of decomposability in, say, life sciences R&D or new product design in FMCG will be the same as in software production. Software architects carefully plan for modular work by specifying interfaces and design rules that allow for this parallelism. They also have the good fortune of dealing with a technology that allows this to be reasonably transparent. With code, what you see is what it does. The rest of us must tread carefully.
It is possible that a day will come when all execution-related activity can be automated (either with robotics or AI), leaving humans free to focus on pure search. This in turn may involve heavy reliance on computer simulations rather than expensive real-world prototyping. On that day, Agile practices might be the norm for every part of every organisation. We just don’t think that day is tomorrow.
Where Agile practices may work well
So, does this mean Agile won’t work for you? No, only that you should be careful. We suggest a “concentric circles” approach to applying Agile, with software development activities at the core. Cautious extension to other search-intensive domains that have many software attributes (low cost of prototyping, parallelism) may come next. Product design, ideation, technical problem solving are good candidates.
But you can do more than just guess whether Agile may work for you. One of the exercises we assign to our Org 2.0 students is to design a prototyping experiment – with properly randomised treatment and control groups – to generate evidence that Agile practices can work in a particular organisational context. One low-cost way to run such an experiment is gamification. Think of an activity that could be run over a few hours or days (and potentially shares critical similarities with your employees’ usual work environment), randomly divide your employees into “hierarchical” and “Agile” teams, and analyse which group did better and why. Our colleague Maria Guadalupe is soon launching a course on how to get Agile right, which should be very valuable too.
The appeal of autonomy
We speculate that the promise of autonomy and hierarchy-free organising (and resulting employee engagement) may also be an important driver of widespread interest in Agile. We have written before about this and our basic viewpoint has not changed: As of today, we still don’t know of any robust, broadly applicable mechanism besides hierarchy for scaling collaboration among a diverse set of interdependent individuals. Yes, there may be more delegation than before, better tools than the simple org chart to design hierarchies, and more attempts to mute the observable differences in status across layers, but the basic multi-layered hierarchical framework is still the one that seems to work in the broadest set of circumstances.
A close examination of prominent exceptions (and there aren’t so many) reveals that they operate under some fairly exceptional circumstances. The idea that we can take an existing set of employees and production technology and seamlessly reorganise them into a non-hierarchical structure has, at this point, almost no evidence to support it. We fervently wish it weren’t so, being egalitarians ourselves. And we should certainly keep trying to find a robust and scalable alternative to hierarchy, because a preference for autonomy and a dislike of inequality are both important and widely held values. But Agile as it stands today is unlikely to be that solution.
Phanish Puranam is an INSEAD Professor of Strategy and the Roland Berger Chaired Professor of Strategy and Organisation Design.
Julien Clément is an Assistant Professor of Organisational Behaviour at the Stanford Graduate School of Business.
-
View Comments
(4) -
Anonymous User
What a interesting discussion! I came here for the article, just to get to know different opinions about Agile and I was presented to such a nice reply to the original paper.
Congratulations to you all!
Anonymous User
"As of today, we still don’t know of any robust, broadly applicable mechanism besides hierarchy for scaling collaboration among a diverse set of interdependent individuals"
I suggest you take a look at the Adaptive Organisation https://agilesphere.eu/whitepaper/adaptive-organisation/
Anonymous User
The document referred contains (yet another?) blueprint of how to make it happen. There are probably hundreds of blueprints of this type, but very little evidence indicates that they work in a sustainable manner, except in very exceptional circumstances.
-
Leave a Comment
Related Reads
Leadership & Organisations
Can We Fix Hierarchy?
Strategy
Strategy as Hypothesis
Strategy
Anonymous User
16/01/2019, 05.54 pm
Thank you, Phanish Puranam and Julien Clément, for a perspective on the Agile hype. As we all know, the results of Agile transformation initiatives are often disappointing. Seven of eight Agile transformation initiatives create little to no value. In the most unfortunate cases, talented individuals are being “inspired” to leave the organization in exasperation. Losing talent is a disaster for digital-age organizations striving to attract and retain top talent.
How come, something so simple to understand as the popular agile frameworks, underpinned with inspiring and proven success stories, cannot be adopted by the “rest of us”?
In my practice as a transformation leader, I have seen a few Agile transformation attempts. Here are some prominent peculiarities of some not so fortunate transformation initiatives:
(MISTAKE 1) Simplistic promises such as “complete twice the work in half the time” are taken literally. This creates the expectation that Agile is an efficiency booster that enables the workers to shoulder “more of the same”. But that is not the way how Agile creates value.
(MISTAKE 2) Agile is perceived as a set of practices that can be rolled out across an organization. That rarely works because the value of Agile does not emerge from pre-defined instructions, rules, organizational structures, roles and processes. Nevertheless, practices are a valuable constituent of Agile. More on that below.
(MISTAKE 3) The agile framework is not being properly adapted to the context to which it is applied. Considering, for example, the product, the organization or the market. Adaptation is hard without understanding the deeper mechanics of the framework. In other words: The WHY behind the framework is not understood. The framework’s prescriptions are followed religiously in a “cargo cult”-manner hoping for the magic to happen.
(MISTAKE 4) A new set of rules and processes is pushed down without having engaged the affected people beforehand. Real participation in decision making didn’t happen. There is little to no buy-in from the workforce. Therefore, the most powerful impact-booster is absent. The general attitude is: “Leadership once again claims to know what is best for all of us”. Which is why the workforce is passively enduring just another change-initiative. Participation, however, is the heart of Agile. Not only in decision making, but also in creating new solutions. In a broader sense, we can call this co-creation. Co-creation thrives where classic decision processes get stuck. Co-creatively crafted solutions can be realized with an impact that push-down change cannot achieve. You could call co-creation the most powerful high-performance tool there is. Without engaged people, this tool cannot be applied. Unfortunately, the popular agile frameworks provide no guidance whatsoever how such an engagement can be achieved. The popular change management approaches, by the way, also have no answer for this. And this is the main reason, why so many Agile transformations fail to get results.
(MISTAKE 5) Agile is indiscriminately applied for all workstreams, regardless of the underlying problem class. That creates unnecessary overhead because problems in the simple domain usually can be solved with established practices and with a predictive management style. Agile, however, thrives in complex problem domains with high degrees of uncertainty. See the Stacey Matrix for example, as a tool for classifying problems.
Leaving the common mistakes aside, what do we see when we examine the available success cases of agile transformations? We usually detect the emergence of new mindsets, a deeply engaged workforce and a raised level of self-management.
Phanish Puranam’s and Julien Clément’s define agile frameworks as a group of rapid, parallel search algorithms using cross-functional teams for product development in an egalitarian fashion. This definition, however, only partly explains the often quantum-leap results of the available success cases.
Furthermore, with Agile approaches, we can create more than just products and services. Business models, organizational structures, processes and rules are being conceived. And implemented and optimized faster using Agile. Customers are being impressed by real business agility. Where flexibility and speed are a core requirement, they are often willing to pay a premium price.
So how can we start a journey to real business agility?
It helps to clarify what we want to change in our organizations. How about this: Let’s improve the way decisions are being made. Let those people decide who are best equipped for making the call. Of course, in co-creation with the tactical and strategic thinkers. What we get are decisions that are: (a) better and (b) executed with more impact.
Decision making in traditional hierarchies suffers from well-known shortcomings such as: sub-optimal coordination across organizational silos, inflexibility / rigidity, single-point information-flow control causing bottlenecks and Chinese Whispers-effects and decision delays. Agile enhances the capabilities of hierarchies in these regards. And without necessarily getting rid of hierarchies altogether.
Agile improves the value streams by focussing all contributors at value creation. Not by poster messages such as “we are … focussed on customer value & take initiative”, which as a mere lip-services achieves nothing. But by instilling new ways of decision making. In the way of co-creation.
How do we do this in practice?
One effective, easy-to-start-with technique is called “Delegation Poker”. Find details in Jurgen Appelo’s “Management 3.0” (Addison-Wesley, 2010). Here is a 5-minute video introduction: https://www.youtube.com/watch?v=VZF-G7MCSG4 A manager “negotiates” with her team who is authorized to make which decision. And how. This is not meant as a one-time event but is being continually adapted to the ever-changing realities.
The Delegation Poker method also demonstrates that Agile is not about getting rid of hierarchies. We still need leadership. For example, for providing rules about how the organization (“the system”) can grow and where the boundaries of self-management are. Some decisions should be delegated to lower levels of the hierarchy. Others shouldn’t. Such clear rules and boundaries are required for responsibility and engagement to grow.
And what is more: Simply by engaging people in Delegation Poker, we showcase a living example of a highly engaging way of collaboration. This evokes engagement, responsibility and courage. And all this happens in a very controlled manner.
Another nice side-effect: invisible electric fences, into which particularly the motivated employees used to run into, disappear.
Delegation Poker broadcasts an important message: “We do take the Agile transformation seriously. Watch us transforming our management work.” This signal encourages the workforce to engage and consider deep change rather than simply complying with leadership’s demands.
Despite its controlled nature, this approach alone already raises the level of autonomy of each single employee. And autonomy is one of the drivers of high performance as described by Daniel H. Pink in “Drive: The Surprising Truth About What Motivates Us” (Riverhead Books, reprint edition 2011).
To make this very clear: The purpose of Agile is not to abolish hierarchy. Agility requires governance. Clear rules and well-managed boundaries of self-management. The purpose of Agile is to better align decisions with the value streams. Also, to execute faster and with more discipline.
In conclusion: While there is usually a decent understanding of the desired benefits of Agile, it remains often unclear how to take an organization to the entirely new level of collaboration and co-creation. Before starting any transformation initiative, we need to set up a proven way to engage the workforce as the very basis of the high-performance we desire. Such engagement cannot be commanded, however. As a good starting point, a method called Delegation Poker can be used to arouse an initial rise of workforce engagement.
There are more holistic approaches available for conducting Agile transformations with a very high success-rate, such as Daniel Mezick’s Open Space Agility. Further advantages are faster and more wide-spread growth of agility. See http://openspaceagility.com/at-a-glance/ We don’t go into detail here.
It remains to be said that Agile is nothing new. At all times organizations have been working in Agile ways. What has changed is this: The rewards for business agility have grown exponentially.
The often cited VUCA-argument, however, is not true for every industry, every market, and every stage of a product’s lifecycle. To ensure that we really need to fundamentally change our way of working, we need to analyse the class of problems and the dynamics of the underlying systems so that we don’t apply excellent tools in a wrong way, which can damage the tool as well as the hand holding it.
Responsibility without passion leads to rigidity. Passion without responsibility rarely gets much done. We need to grow both passion and responsibility. Engaging the people is key here. Agile is more of an outcome than a method of achieving this. Beware of Agile frameworks that claim that it is sufficient to follow its rules by the book to create a high-performing organization.
Finally, best success to Maria Guadalupe with launching a course on how to get Agile right. I am excited to learn more about this.
Apologies for a comment that is longer than the underlying article. I hope some of these thoughts can help the reader to start her journey to explore authentic agility.