Beware ‘Experts’ Who ‘Beat Odds,’ Suggests Research

Experts who go against the grain and pull off a big win are not the ones to follow – it was just a fluke after all. At least that’s how Jerker Denrell, Professor of Behavioural Science at Warwick Business School, along with Christina Fang, of Stern School of Business, see matters. The two looked at three years of experts’ quarterly forecasts for interest rates and inflation printed in the Wall Street Journal and found some surprising results.

Those well-known experts who had pulled off a big windfall by going against the tide and winning were, over the long term, the worst at forecasting. Professor Denrell extends this to entrepreneurs who make it big, and are then seen as the example to follow, when his research suggests there are many other factors involved, and it was probably just came down to luck.

Instead, the lesser-known, conservative experts, who picked the predictable numbers, were the ones that came out on to, and hence they are the ones investors and the media should be following.

“People who hit it big going against the grain are the ones we turn to for wisdom on future events, but our results revealed they tend to be bad at making forecasts,” said Professor Denrell in the Harvard Business Review.

“People who have decent but not exceptional performance with predictions are the ones with the most to offer us,” he continued. “We tend to admire these people because we remember that they hit and no one else did, they separated themselves from the pack and it worked out. But the very traits that lead them to make theses out-of-sync predictions usually also lead to misses.”

Professor Denrell said he followed up the findings with experiments in lab conditions, which confirmed his theory.

“In the lab people with that sort of personality, that tolerance for risk and for being non-conformist were the ones that made the wild predictions,” said Professor Denrell. “The underlying idea is that predicting something unusual is not that informative. We houldn’t rely on people who focus on unusual events, but we do.”

In the study of the data from the Wall Street Journal, Sung Won Sohn was one of the few who correctly predicted a high inflation rate when the consensus forecast was low. He credited this with visiting a California jeans manufacturer who he met and told him he couldn’t keep up with demand for his $250 jeans. But in the preceding two periods Sohn was ranked 43 and 49 out of 55 forecasters.

“You can see this again with entrepreneurs. They enter an industry and hit it big with one idea. It’s human nature to see that as impressive and to seek that person’s counsel on another venture, but that could be his one hit among many failures – and most likely is,” said Denrell. “Yet he might get a king’s ransom to do the same in another industry. Top performance and outcomes are often produced by things besides skill, so we shouldn’t reward them as much as we do.”

The results suggest that successful bold predictions don’t signal the competence of the predictor at all, said Denrell. “Even if someone makes two successful bold predictions, it’s more likely a fluke.”