The shift to virtual working has produced a data boom that could revolutionise organisations – if they know what to do with it.
This blog post is based upon our new digital publication The Organisational Analytics eBook: A Guide to Data-Driven Organisation Design. You can read more about it below, or download it for free here.
Covid-19 has dealt most businesses a heavy blow, but the pandemic has at least one underacknowledged upside. By moving organisations from the office into the virtual space, Covid-19 has cracked open a treasure trove of data that can be used to streamline and optimise how organisations operate. We wrote a (free) eBook to help you capitalise on it.
The inner workings of organisations used to be largely invisible. Before we all started working from home, pivotal decisions were made around a conference table, at a restaurant over lunch or even in the lift – where interactions are hard to track. But virtual work, largely conducted via digital platforms such as Zoom and Microsoft Teams, makes the intangible concrete. Chat logs, video recordings, activity trails on collaborative projects, etc. form a comprehensive, real-time record of organisational activity, offering managers new levels of insight into everything from employee morale to how informal ties among employees affect the outcomes of business decisions.
Clearly, this is sensitive information. It is crucial that organisations uphold high ethical and legal standards on data privacy and compliance; putting in place a clear data governance framework is a must. Data need to be appropriately anonymised and aggregated at a sufficiently high level to guarantee privacy for employees. This requires some work, but these data can yield powerful discoveries.
The rise of data-driven organisation design
It is well known that data and algorithms can help companies achieve specific business objectives, such as improving the results of marketing campaigns or streamlining hiring by “crawling” candidates’ CVs for desired keywords. But there is much, much more you can do beyond streamlining hiring and forecasting employee churn: Your entire approach to organisation design can be data-driven. To help non-techie business practitioners understand these opportunities and what it means for their role, we wrote The Organisational Analytics eBook: A Guide to Data-Driven Organisation Design. It is available at no cost; download it here. We cover use cases that range from improving diversity and inclusion, resolving bottlenecks in decision making and breaking silos, all the way to measuring culture.
The book’s core premise is that, while organisations are highly diverse and ever-evolving, their basic problems are perennial and universal. They always come down to issues of division of labour and integration of effort: breaking large objectives down into smaller tasks and reassembling the results into a coherent whole. If organisations can learn to use data and algorithms to manage these issues better, their performance will substantially – and sustainably – improve. This is true of small teams, large companies and government agencies alike.
Three types of analytical methods
The analytical methods used in data-driven design fall into three categories (explored in more depth in this INSEAD Knowledge post): perception, prediction and prototyping. Perception involves building an accurate picture of what is actually going on in your organisation by relying on data rather than assumptions. It’s the least sophisticated step, but it can often make a big difference. For example, good data can show where vital information is falling through the cracks between silos. Repair the communications breach, and your problem is solved.
Not all organisational challenges are so cut-and-dried, however. More complex questions may require moving from perception to prediction – predicting future outcomes, possibly through machine learning. We explain when this is feasible and when it can pay off. Algorithmic predictions, of course, are only as good as the data from which they extrapolate. Prototyping doesn’t depend quite so directly on past data: It lets you generate your own data by carrying out experiments. Through randomisation, experiments create control and treatment groups that allow for synchronous comparisons. We also introduce the idea of gamified prototyping of new organisation designs. This often gives us much of the insight we would get from a full field experiment at a fraction of the cost.
Three types of organisational issues
It can help to structure your analytical efforts into different types of organisational issues that can be solved through data and algorithms. We lay out such a structure in the book. We think of organisation design as involving three levers, or three groups of decisions which can all benefit from a data-driven approach, though often with different tools.
Structuring involves the use of authority, incentives and grouping to solve issues of coordination and cooperation. For instance, are silos in your organisation preventing synergies? By collecting better information on workflows and on employee interactions, organisations can identify “hot spots” where coordination failures are happening. We show how to use network analysis in order to do just that.
Sorting revolves around “who’s in and who’s out” at the organisation, department or team level. This includes questions around hiring, self-selection within projects and the increasingly urgent matter of workplace diversity. It goes without saying that comprehensive workforce data are critical to addressing these issues. The book provides detailed use cases that show why better data yields great value in this area.
Sensemaking groups together issues around leadership, vision and culture. This is perhaps the lever most affected by the Covid-19 shift to virtual working. Information gathered through Slack and Zoom form detailed snapshots of corporate culture in action. These platforms (and others like them) are ideal forums for experimental intervention. Experimenting with new group processes, gathering sentiment data or impressions on the usefulness of a meeting can all help improve design. For example, even while maintaining individual anonymity, it is possible to aggregate text data and use machine learning techniques to identify the values or concepts that are important to people based on the words they use. Even public websites displaying employee reviews can provide valuable insights into corporate culture which are increasingly used by companies.
An evolving document
The field of data-driven design is far from static – in fact, it is growing all the time. To reflect this, we will be updating our book frequently with the latest advancements in the field, just as developers are continually refining their software and releasing new versions. To receive the latest edition as soon as it comes out, please add your name to our free mailing list. We also welcome any feedback from readers that may help in shaping future iterations of this book.
The data boom following Covid-19 may hold the seeds of rapid recovery for many companies – and help build more resilient organisations going forward. The tricky part will be sifting through the data deluge to find what’s most relevant, and strategically applying that information with rigorous methods. We intend this book to be a freely accessible resource to help leaders get started.
Phanish Puranam is the Roland Berger Chaired Professor of Strategy and Organisation Design at INSEAD.
Julien Clément is an Assistant Professor of Organisational Behaviour at the Stanford Graduate School of Business.
Don’t miss our latest content. Download the free INSEAD Knowledge app today.