Jump to content United States-English
HP.com Home Products and Services Support and Drivers Solutions How to Buy
» Contact HP

HP.com home

Predicting the future -- with games

How researchers are using the lab to forecast business futures


HP Labs

» Research
» News and events
» Technical reports
» About HP Labs
» Careers @ HP Labs
» People
» Worldwide sites
» Downloads

back of business man watching digital display at stock exchange
Content starts here
Researchers are testing the service with a customer. It's also attracting interest from major corporate players in communications, software and energy exploration.

By Simon Firth, July 2006

Predicting the future isn't just the province of fortune tellers or crystal-ball gazers. A team of researchers at HP Labs is doing just that – with the tools of economics.

Labs’ most advanced project in this area is called BRAIN.

“That stands for Behaviorally Robust Aggregation of Information in Networks,” says researcher Leslie Fine, an experimental economist at HP Labs and part of a team that is using laboratory experiments with human subjects to evaluate economic propositions. The program, founded in 1994 by researcher Kay-Yut Chen, has been nationally recognized in applying this kind of research to the problems faced by corporations.

Fine, Chen and HP Senior Fellow Bernardo Huberman developed BRAIN to tackle a problem that troubles the corporate world – how to extract accurate information about future events (such as predicted quarterly revenues or expected demand for a product) from small teams of knowledgeable workers.

Market mechanisms

In the wider world the best mechanism for predicting the future is to run a market. When large numbers of people speculate on stocks or commodities, for example, their prices reflect everything that is currently known about how they might perform in the future.

The University of Iowa’s famous Iowa Electronics Market has shown that large groups of people can also predict electoral results and the future direction of major economic indicators with almost unrivaled accuracy.

Markets don't work well with small groups of people, though. It’s not that such groups aren’t knowledgeable, but outcomes from small markets can easily be skewed thanks to the low number of players offering bids, or through deliberate manipulation.

Working with small groups

But without markets, the decisions made by small groups are equally open to manipulation. Take an IT procurement team trying to predict the future price of a key component. Typically, Fine explains, they’ll do this by “sitting around a room once a month and yelling at each other until a number comes out.”

As a result, she says, “he who yells loudest wins.” Quieter but more knowledgeable team members -- or team members too intimidated to contradict the boss -- don’t get heard and the predictions suffer as a result.

BRAIN gets around these problems in three ways: It makes the prediction process anonymous; it asks people to back up their predictions with real money -- essentially, to make bets on where they think the numbers will land; and it makes the whole exercise a game.

Determining risk tolerance

If you are asking people to make bets, you need to know their appetite for risk in order to makes sense of the bets they place.

So BRAIN first has the team play a market game which requires them to put stakes on various imaginary outcomes. From that, researchers calculate a behavioral coefficient for each player that summarizes their risk attitude and their predictive power.

Then players are asked to make bets on the real question in play (such as 'how many handheld devices will we sell this month?'). According to how accurate their bets turn out to be, the players receive a payoff -- typically in tens rather than hundred of dollars.

Meanwhile, says Fine, “we weight the bets by the coefficients we have for each player, aggregate them, and pass back the prediction.”

BRAIN beats traditional methods

Such a mechanism, it turns out, works extremely well. Because it gets people to divulge what they know, BRAIN’s predictions consistently beat forecasts generated in more traditional ways. It helps that people generally enjoy playing BRAIN, too. It means they'll go back and use the tool again.

The work that went into BRAIN also suggests ways to mitigate against other business problems -- such as sandbagging -- where individuals underestimate their future performance so as not to lose out on their pre-agreed bonuses.

In the face of that, says Chen, a manager could simply pay a bonus to people who made accurate forecasts of their performance. But offer such a deal to some people, he adds, “and you'll get a very accurate forecast, but they won't sell anything!”

Getting accurate sales predictions

To solve this, researchers have proposed presenting salespeople with the option of receiving more of their pay as fixed compensation and less as bonus, or vice versa.

“So then you say to the salesperson,” says Huberman, “How would you like to get paid?”

“If he knows he’s going to sell everything,” Huberman suggests, “he’ll want all bonus. If he knows he won’t sell anything, he’ll want to bring a check home. So by the way he chooses, you can figure a probability that he will actually sell a certain number of pieces.”

As a result, the boss knows the true sales outlook and the salesperson is still well compensated for being good at his or her job.

Tests in the lab

Huberman and Chen were able to successfully test this idea in their experimental economics lab, where people played a series of games that modeled such a sales situation.

It’s unusual for a corporation to have such a research facility, but very useful.

Several groups within HP have already engaged Chen to design games to help them reformulate the contracts they sign to realign them in the interests of both signatory parties.

“Contracts are a bit like the rules of a game,” says Chen. “If you change the rules of the game, people will play differently. And once you understand how, you can write better contracts.”

Customer testing

BRAIN has shown that economic games have predictive power outside the lab, says Fine, who ran a successful field test with a financing forecast team in HP Services.

“At the beginning of each month, we would predict operating profits and revenues for the end of that month,” she says. “And we blew their numbers out of the water.”

Fine is running a pilot of BRAIN with its first external customer. The service is also attracting interest from major corporate players in communications, software and energy exploration.

BRAIN can be useful, says Fine, “any place you have a small team that has some kind of specialized knowledge that you want to tap, and where you can take what you want to know and rephrase it as a bet.”

Shorter time scales work best. “No one wants to place bets on things they’re going to find out about in ten years,” Fine notes. It’s also important, she says, that “there’s no ability to lose money here. So we’re not gambling.”

Other prediction mechanisms

"This effort," says Huberman, "is really about harvesting implicit knowledge in organizations." He has pioneered mechanisms for identifying the best set of people to poll for such knowledge -- groups that may not always echo formal organization charts.

Researchers have developed other tools for prediction as well. One technique is aimed at helping risk-averse business managers make decisions and another helps businesses better predict demand. (See 'Afraid to make risky business decisions?' and 'Riding the peaks'.)

Tools like these are growing increasingly important, Huberman argues, as we're faced with burgeoning amounts of information.

“It’s very hard to attend to what is important and what is salient or not," he says. "This offers us a way to know.”

Simon Firth is a writer and television producer living in Silicon Valley.

Related links

» Related research: BRAIN
» Information Dynamics Lab

News and events

» Recent news stories
» Archived news stories

digital read out board

Research paper

» Forecasting uncertain events with small groups

Screen shot of inventory

Printable version
Privacy statement Using this site means you accept its terms Feedback to HP Labs
© 2009 Hewlett-Packard Development Company, L.P.