As organisations take their first halting steps towards recovery in this global crisis, they may find that the innovative ideas or capacities they most need aren’t available in-house. They will need to look outward for help from partners and other stakeholders – including their customers and fans. Crowdsourcing competitions may be especially worth considering as part of an open innovation toolkit. By widening the pool of contributors well beyond the focal firm, crowdsourcing compensates for any limitations or biases that may constrain the firm’s innovative vision.

However, it is difficult to get it right. What you get out of it is highly dependent upon what you put in – i.e. how you design the contest itself. Arbitrary or haphazard parameters may reduce the value of the submissions you receive. Ideally, contests should be organised based on your specific objective. Is your primary concern quantity or quality? Are you looking for one “best” solution, or a pool to choose from?

Although contests have been widely used across industries ranging from data science and medicine to architecture and design, limited guidelines exist for how to tailor rules and incentives to intended outcomes.

In a new project, recently published in the RAND Journal of Economics, I develop a data-driven framework to measure the impact of different incentive structures on participant behaviour in contests. With some customisation, this framework can be used by contest organisers to measure imbalances in their contests and simulate the likely impact of alternative prize structures and entry restrictions, before running the actual contest. The key is that organisers can obtain more value from their contests without increasing the contest budget, simply by changing the prize distribution to encourage healthy competition.

Rewriting the rulebook

For the study, I partnered with an online platform that runs ideation contests for major brands such as Google, AT&T and General Electric. The platform shared data from 181 real-life contests launched in 2011-2015, with a total of 8,875 participants. The dataset included all submissions in their entirety, how each one was rated by contest judges and the design of each contest. In addition, I analysed browsing data from the platform to measure the number of non-entrants for each competition, i.e. people who visited the relevant pages but decided not to contribute. We can assume that they opted out because, in their minds, the cost of participating outweighed the probable benefits. Presumably, many believed they were unlikely to win.

My intuition was that this attrition would be a loss to contest organisers to the extent that the pool of participants were highly differentiated in their skills. If less technically skilled users were discouraged from participating, the resulting entries would be restricted to a smaller group of experts with similar backgrounds. They would be long on technical prowess, but short on diversity. In cases where companies were looking for state-of-the-art and outside-the-box solutions, they would receive fewer entries fulfilling both requirements.

Using an empirical model adjusted to fit the contest dataset, I ran a number of virtual “counterfactual” simulations. I found that by increasing the number of potential prizes and reducing the proportion of winnings that can go to any one participant, organisers could shift the cost-benefit analysis of less-skilled users who might otherwise opt out. This would create more healthy competition for the elite entrants. At the same time, giving out prizes too generously diminishes their value, removing the incentive for participants to work hard.

The case of deepfakes

Let’s look at a practical example. Earlier this year, several large organisations, including AWS, Facebook and Microsoft, co-hosted a global contest for building algorithms to detect deepfakes on Kaggle, a platform for data science. The contest provided a community of computer vision experts and machine learning hobbyists with a massive 470 GB dataset of deepfake and real videos, and attracted submissions from over 2,000 teams. The sponsors allowed participants to submit at most two entries per day, and promised a total prize of US$1 million, which was split across the top five winners in the following way:

First prize: $500,000

Second prize: $300,000

Third prize: $100,000

Fourth prize: $60,000

Fifth prize: $40,000

Why five prizes? Why two submissions per day? And why this particular distribution, with 80 percent of the money reserved for the top two winners?

According to the model, most contest organisers would experience a better end result, in terms not only of volume but also of overall quality of submissions, if they vastly increased the number of winners to 20-30 percent of the total amount of anticipated entries, assuming participants could win only one prize each. Lowering the bar only a little bit, e.g. by offering three or four prizes instead of two, did not meaningfully alter the cost-benefit calculation by which the curious are converted into active participants. Placing limits on the number of submissions per participant – another method of levelling the playing field – can help encourage timid users to try their luck, but the trade-off is a reduction in the quality of submissions.

In sum, contest organisers should look carefully at their structure and incentives, aiming for a steady balance of the aspirational and the accessible. In so doing, they will make the most of the limited resources available to them in this crisis.

Pavel Kireyev is an Assistant Professor of Marketing at INSEAD.

Don’t miss our latest content. Download the free INSEAD Knowledge app today.

Follow INSEAD Knowledge on Twitter and Facebook.