Why Experts are so Wrong

The following is an excerpt from Jackass Investing: Don't do it. Profit from it.. It's a book that takes "conventional knowledge" on investing and argues that it's a myth.

A truly telling study conducted by Philip Tetlock, a research psychologist at Stanford University, helps to answer this question. The study looked at 82,361 predictions made by 284 experts (pundits).

Tetlock asked the pundits to rate the probability of three potential outcomes for each prediction they felt qualified to make. In the case of an economic expert, he may ask, for example, whether the economy was likely to:

• remain at the same level of growth
• grow faster
• grow slower

What he found was that the experts could literally have been beaten by dart-throwing monkeys. The experts’ predictions were worse than if they had randomly selected the outcome.

And it gets worse. Tetlock asked similar questions to those who were not experts. The experts scored no better than did this non-expert group. Their massive level of knowledge relative to the non-experts did nothing to improve their predictive capabilities. Tetlock wasn’t the first to discover this. In one earlier study from the 1960s, researchers asked college counselors to predict the grades high school students would achieve as college freshman. The counselors were provided with test scores, grades and the results of personality tests. They were also permitted to interview the students. Their results were compared to those derived from a formula based solely on test scores and grades. The outcome was that the counselors were beat by the formula.

An even earlier study from the 1950s involved the results of tests used to diagnose brain damage in patients. This data was presented to a group of clinical psychologists and their secretaries. The result of the study revealed that the psychologists’ diagnoses were no better than the secretaries’.

The fact is that people over-think and quickly form biases based on limited information. Once that bias is formed, substantial effort is employed in supporting it, regardless of whether it is right or wrong. People really hate to be wrong.

Rats!

Right before Christmas, in 2009, my wife was searching for the perfect surprise gift for our three boys. She ended up buying three “Dumbo” rats from the local pet store. (The name “Dumbo” refers to the rats’ big ears, not their intelligence.) While we were told by the salesperson that the rats were smart and could be trained, no one on earth would ever assume that a rat could outsmart a human in any given situation. But in the basic understanding of probability, without-a-doubt a mandatory skill for achieving any measure of success in money management, rats seem to be able to outperform people. Here’s an example that Tetlock witnessed at Yale University 30 years before he published the results from his pundits study.

In this particular Yale study, a rat was placed in a T-shaped maze. The researchers placed food in the left part of the “T” 60% of the time and in the right part 40% of the time. Students were asked to predict on which side of the “T” the food would appear each time. The rat, of course, was left to find the food on his own. The students weren’t told that there would be a bias to one side. But it was the rat who eventually figured out that the food was more likely to appear on the left side than the right and, as a result, almost always went to the left first, scoring roughly 60%. In contrast, the students scored only 52%!  In trying to outsmart the placement of the food, the students seemed to be looking for patterns that clearly didn’t exist and, as a result, were outsmarted by the rat.

This is a common human behavior. We try to outsmart the system (or the market) looking for patterns that don’t exist – desperate to “beat the system.” When that desire is combined with our need to be ‘”right” and our easily established biases, we can become dumber than a rodent. Experts – pundits and advisors – surprisingly enough, are (in most cases) humans too.

This leads us to a third significant discovery made by Tetlock: the more often an expert appears on TV or other media, the worse his or her batting average. Think about that.

The experts who are most often touted, and who reach the most disciples, are shown to be the most often wrong.

This is not surprising. Experts exist to provide the media with a steady flow of content and, more importantly, entertainment. Their purpose is not to provide you with useful, profit-making information. Furthermore, if an expert has staked his reputation on a prediction announced to millions of fans re-peatedly on television, across the Internet, over the radio, and in print, it will be very difficult for him to have a change of mind – even if the evidence overwhelmingly indicates he is wrong. At that point he is locked in to his bias.

If there was ever a reason to avoid expert opinions, this is it.

“One small step for a man. One giant leap for mankind.”

July 20, 1969 was a huge day for me. As a young boy on a fam¬ily vacation, I remember sitting in my grandparents’ living room in Ohio watching the grainy, small-screen television images of the first humans walking on the Moon. I had always been a space buff. I still vividly recall being totally enamored with the Mercury missions and the hero/astronauts who flew them, Alan Sheppard, John Glenn, Scott Carpenter, Gus Grissom, Wally Schirra, Gordon Cooper, and Deke Slayton. Starting with the Apollo program, I had made scrap books for all the flights and my brother and I even rigged up a tape re¬corder to the television to record virtually every telecast of every flight. I still have the tapes today.

The Apollo flights were dramatic from the start. Tragedy struck the U.S. space program for the first time when, in January 1967, the Apollo 1 fire killed Gus Grissom, Ed White and Roger Chaffee during a training accident on the pad at Kennedy Space Center. But the resilience of the U.S. space program was impressive. Less than two years later, Frank Borman, James Lovell and Bill Anders orbited the Moon in Apollo 8. Their Christmas Eve telecast, where they read the first 10 verses from the book of Genesis, was the most watched television program ever up to that time.

NASA had worked hard to earn their newly acclaimed suc-cess. The U.S. space program was an endeavor of immense proportion. More than 400,000 people had been working on the program over a 10-year period leading up to the lunar land-ings.  Finally, after this decade of patient, steady advancement in space flight technology, punctuated by triumphs and catastro-phe, Neil Armstrong and Buzz Aldrin climbed down the ladder of the lunar module to stamp the first human footprint on the moon. …Or did they?

I’ve made up my mind – don’t confuse me with the facts

An otherwise rational guy I know is absolutely convinced that the Apollo moon landings were fake. He’s positive that NASA never reached the Moon. He’s not alone. Depending on where you live, between 6% and 28% of your neighbors side with him. These are the results of polls taken over the past 20 years in places ranging from the U.S.A. to Russia. In a 2009 poll conducted by the British Engineering & Technology magazine, 25% of Britons stated they did not believe humans had ever walked on the Moon.

These people cite numerous pieces of evidence in support of their claim that the Moon landings were a hoax. All of these have been re¬futed numerous times by qualified experts, but the “hoax” proponents continue to thrive. And it doesn’t take much to gain new converts to their cause. In 2001, public skepticism of the validity of the Moon landings increased from 6% to 20% after Fox Television aired a TV show titled Conspiracy Theory: Did We Land on the Moon? In 2000, 28% of Russians polled stated that they did not believe that Americans walked on the Moon.

Despite the solid evidence refuting these hoax charges, de-spite the significant weight of evidence supporting the validity of the Moon landings, despite the enormous implausibility that the 400,000 people who worked in the U.S. space program and the 24 astronauts who went to the Moon, including 12 who walked on its surface, could join together in an elaborate conspiracy to defraud the rest of the world’s population – and keep it quiet – millions of people still believe the Moon landings were a hoax.

People have an enormous capability for ignoring the facts and believing what they have already made up their minds to believe. This capability seems to also hold true with their choices of how to manage their money.

Compounding the problem

Everyone has a bias, which is formed through a combination of research, learned reasoning and intuition. Each bias will be reinforced when you seek out experts, as you will welcome new information that supports your bias and dismiss information that conflicts with your view.

A major problem with following expert advice is that it compounds an individual’s bias.

People form their biases and then only welcome the views of those experts who hold the same biases. And those experts in turn dismiss new information that doesn’t fit in with what they already believe. As a result, people quickly reach a tipping point where their minds are set and they are locked in to their view.

Another friend of mine is a gold bug. She devours any in-formation that supports her view that gold is going up in price. The fact that gold has risen in price for much of the past 10 years only supports her belief in the truth behind this information and the experts who are presenting it to her. But that doesn’t make the information correct. And it certainly doesn’t mean that the result of that information will be higher gold prices going forward. But it does provide her comfort in pursuing her goal of owning a significant amount of gold.

When people get locked in to a view and only consume information that supports their view, they are no longer engaged in the pursuit of profits, but the pursuit of entertainment. All of the studies described above indicate that my friend’s fascination with gold and her acceptance of expert opinion that supports her view will lead to a loss of money. But following “expert” advice becomes a drug. It’s hard to stop. The best advice is not to start.

Comments

[Jacob Braaten]

Great post. This is absolutely true... once people have made up their minds one way or the other, it can be very difficult to get them to change their minds no matter what the evidence supports. Very few people will admit to being wrong even if they know they are!

[Mike Hunt]

I think the rat in the T maze can outperform a human because of the superior sense of smell.

-Mike

[Apex]

@Mike,

I don't think so.

That would be a bit of a silly test if the food was already there for the rat to smell.

The point was prediction. It doesn't say the rat had to choose before the food was presented but it does say that the students did poor at prediction, and that the rat ALMOST ALWAYS went left meaning the rat got it right 60% of the time. If it was using smell it should have gotten it right nearly 100% of the time by going the direction of the smell each time but it did not. It almost always went left.

That means the rat eventually was able to perform a low level computation that figured out the probability was higher to the left so that is where it went to wait for it's food. This would make sense in the wild because rats that were good at predicting where food was more likely to be, would out survive rats that are poor at it. The rats are not thinking. They are simply doing some kind of repetition meter that was able to detect the direction that repeated more often.

As humans we think we are pretty smart so we are looking for more complex patterns or clues to tell us where it will be next. And often our "thinking" is downright stupid like the boards showing the last 15 colors at the roulette table in Vegas. Our smart brains say if it has been black the last 7 times it is way over due for red and we smartly over bet on red. And we are of course wrong because mutually exclusive probably events have no impact on future events. A rat would likely figure that out. Many of us humans don't because we think we are thinking but our thinking too often turns out to be horribly flawed such that it leads us to conclusions that are worse than if we did no thinking at all.

Of course I don't think this way and I know this because I think it. :)

[Rick Francis]

@Apex

You make a good point- there are many situations where there is NO PATTERN.

We have to recognize that our minds try to find patters even when they don't exist. Then use a higher level of thinking to admit we can't predict the next spin of the roulette wheel from past results and similarly we can't use technical analysis to predict the future stock price from any chart of the past prices.

-Rick Francis

[Old Limey]

The velocity and trajectory of a body remains constant unless the body is acted upon by an external force (Sir Isaac Newton's First Law of Motion).

You cannot predict what the stockmarket, or a fund, or a bond will do tomorrow, next week, next month, or next year. However you can study its trend and use the direction and strength of the trend to guide your decisions. You don't even need to know what external forces are causing the trend to change. You just have to base your Buy, Sell, or Hold decisions opon your analysis of the trend. Low volatility investments have smoother trends and are a lot easier to work with.

There's an old trader's saying, "The Trend is your Friend". This is why smart, active investing is a whole easier than most people think and why only fools make predictions.

[Kelly@All-FinancialNews]

Selective exposure and selective retention are parts of the human psyche that we are all guilty of acting upon. We seek out information that agrees with our already preconceived ideas because it is cognitively uncomfortable for us to receive information that disagrees with our thoughts and beliefs on a subject. Not only this, but when we are presented with information from all sides of an argument, and are later asked about this information, we will most certainly only relay back the information that we ourselves agree with. This must play a major role in more facets of our lives than anyone can even be aware of.