illusion of skill

The illusion of skill refers to the belief that skill, not chance or luck, accounts for the accuracy of predictions of things that are unpredictable, such as the long-term weather forecasts one finds in farmers' almanacs and the predictions of market gurus about the long-term ups and downs of the stock market. The illusion of skill also accurately describes the apparent accuracy of remote viewers. Given all the guesses remote viewers make about what they claim to see telepathically, chance alone would account for some of those guesses being somewhat accurate. Much of the accuracy ascribed to remote viewers, however, is due to the liberal and generous interpretations given by themselves or "experts" of their vague and ambiguous descriptions of places or things. Also, subjective validation accounts for the illusion of skill of "experts" in such fields as palm reading, mediumship, astrology, and criminal profiling.


Stock gurus--people who predict the rise and fall of the price of stocks and have large numbers of people who act on their predictions--are essentially part of an entire industry "built largely on an illusion of skill" (Daniel Kahneman, Thinking, Fast and Slow, p. 212). No market guru has gone broke selling advice, however, despite the fact that market newsletters are--in the words of William A. Sherden--"the modern day equivalent of farmers' almanacs" (The Fortune Sellers, p. 102). In 1994, the Hurlbert Financial Digest found that over a five-year period only one out of 108 market-timing newsletters beat the market. You might think that that one did so because of skill, but you'd be wrong. Chance alone would predict that more than one out of 108 would beat the market. If the one who got it right could do so again and again, you might have a case that some skill was involved. But, the fact is, nobody consistently beats the market. The reason is that the market is inscrutable. Given the number of people predicting the rise and fall of stocks and the stock market as a whole, a few are likely to hit the mark once in a while just by chance. One or two might even make accurate predictions several years in a row. It would take decades, however, to get enough data to reasonably conclude whether the string of correct predictions was more likely due to skill than to chance. On the other hand, it can't be denied that several market gurus have such significant followings that their predictions can become self-fulfilling prophecies. The prediction of a market rise or fall by one of these gurus can trigger a massive sell-off or buy-in and cause the market to rise or fall as "predicted."

"The stock market is a psychological soup of fear, greed, hope, superstition, and a host of other emotions and motives," according to Sherden. The market doesn't follow a set of consistent or rational laws. The market's volatility is due to its being a complex system with both rational and irrational forces at work (Sherden, 118). Daniel Kahneman writes:

Some years ago I had an unusual opportunity to examine the illusion of financial skill up close. I had been invited to speak to a group of investment advisers in a firm that provided financial advice and other services to very wealthy clients. I asked for some data to prepare my presentation and was granted a small treasure: a spreadsheet summarizing the investment outcomes of some twenty-five anonymous wealth advisers, for each of eight consecutive years. (Thinking, Fast and Slow, p. 215)

Using 28 correlation coefficients, Kahneman found that the "results resembled what you would expect from a dice-rolling contest, not a game of skill." He told the executives of the firm that it was rewarding luck as if it were skill. They saw the data and couldn't deny it, but they ignored the findings. "Facts that challenge such basic assumptions—and thereby threaten people’s livelihood and self-esteem—are simply not absorbed," wrote Kahneman. "The mind does not digest them."

The illusions of validity and skill are supported by a powerful professional culture. We know that people can maintain an unshakable faith in any proposition, however absurd, when they are sustained by a community of like-minded believers. (Kahneman, p. 217) 

A study done by researchers at Duke University indicates the depth of the problem of these illusions of skill and validity rampant in the corporate world.

For a number of years, professors at Duke University conducted a survey in which the chief financial officers of large corporations estimated the returns of the Standard & Poor’s index over the following year. The Duke scholars collected 11,600 such forecasts and examined their accuracy. The conclusion was straightforward: financial officers of large corporations had no clue about the short-term future of the stock market; the correlation between their estimates and the true value was slightly less than zero! When they said the market would go down, it was slightly more likely than not that it would go up. These findings are not surprising. The truly bad news is that the CFOs did not appear to know that their forecasts were worthless. (p. 261)

Kahneman wryly comments: "An unbiased appreciation of uncertainty is a cornerstone of rationality—but it is not what people and organizations want" (p. 263).

It's probably not fair to pick on one individual to illustrate the illusion of skill, but the case of Elaine Garzarelli is too salient to pass up. She became famous overnight for predicting Black Monday (the collapse of the stock market on October 16, 1987). At the time she was a research analyst and money manager at Shearson Lehman Bros. She based her prediction on fourteen monthly indicators. Just four days before Black Monday, Garzarelli announced on Money Line her prediction of "an imminent collapse in the stock market." Business Week described her prediction as "the call of the century." After the collapse, she continued to predict that the Dow would drop even lower. What she didn't predict, however, was that the market would quickly rebound. Her indicators failed her more often than not. Sherden analyzed her predictions about the market going up or down from 1987 to 1996 and found she was right 38% of the time. She'd have done better flipping a coin. Her fame and fortune were made, however, based on one lucky guess. The mutual fund she managed was eventually shut down and she was fired by Shearson Lehman, but her luck did not give out. She's still at it and has recently let the world know that she is bullish.

Should investment advisers be avoided at all costs? Not necessarily. Sherden writes: "The best thing that an investment adviser can do for you is to assess your financial needs and design a tailored investment program to meet them. A good financial adviser will consider your tax situation, cash flow needs, and risk tolerance."

If your financial adviser tells you that she has figured out the market and if you follow the advice she'll sell you she guarantees you'll make a fortune, remember this little blog about the illusion of skill.