"Survivorship Bias"

That's the title of a fascinating article by David McRaney at his Web site 'You Are Not So Smart'.  He discusses statistical analysis in World War II, shows how a very bright statistician prevented the US Army Air Force from making a disastrous mistake, and demonstrates how that illustrates a problem in decision-making that affects us all.

Simply put, survivorship bias is your tendency to focus on survivors instead of whatever you would call a non-survivor depending on the situation. Sometimes that means you tend to focus on the living instead of the dead, or on winners instead of losers, or on successes instead of failures. In Wald’s problem, the military focused on the planes that made it home and almost made a terrible decision because they ignored the ones that got shot down.

It is easy to do. After any process that leaves behind survivors, the non-survivors are often destroyed or muted or removed from your view. If failures becomes invisible, then naturally you will pay more attention to successes. Not only do you fail to recognize that what is missing might have held important information, you fail to recognize that there is missing information at all.

You must remind yourself that when you start to pick apart winners and losers, successes and failures, the living and dead, that by paying attention to one side of that equation you are always neglecting the other. If you are thinking about opening a restaurant because there are so many successful restaurants in your hometown, you are ignoring the fact the only successful restaurants survive to become examples. Maybe on average 90 percent of restaurants in your city fail in the first year. You can’t see all those failures because when they fail they also disappear from view. As Nassim Taleb writes in his book The Black Swan, “The cemetery of failed restaurants is very silent.” Of course the few that don’t fail in that deadly of an environment are wildly successful because only the very best and the very lucky can survive. All you are left with are super successes, and looking at them day after day you might think it’s a great business to get into when you are actually seeing evidence that you should avoid it.

. . .

Wiseman speculated that what we call luck is actually a pattern of behaviors that coincide with a style of understanding and interacting with the events and people you encounter throughout life. Unlucky people are narrowly focused, he observed. They crave security and tend to be more anxious, and instead of wading into the sea of random chance open to what may come, they remain fixated on controlling the situation, on seeking a specific goal. As a result, they miss out on the thousands of opportunities that may float by. Lucky people tend to constantly change routines and seek out new experiences. Wiseman saw that the people who considered themselves lucky, and who then did actually demonstrate luck was on their side over the course of a decade, tended to place themselves into situations where anything could happen more often and thus exposed themselves to more random chance than did unlucky people. The lucky try more things, and fail more often, but when they fail they shrug it off and try something else. Occasionally, things work out.

. . .

You succumb to survivorship bias because you are innately terrible with statistics. For instance, if you seek advice from a very old person about how to become very old, the only person who can provide you an answer is a person who is not dead. The people who made the poor health choices you should avoid are now resting in the earth and can’t tell you about those bad choices anymore. That’s why it’s difficult not to furrow your brow and wonder why you keep paying for a gym membership when Willard Scott showcases the birthday of a 110-year-old woman who claims the source of her longevity is a daily regimen of cigarillos, cheese sticks, and Wild Turkey cut with maple syrup and Robitussin. You miss that people like her represent a very small number of the living. They are on the thin end of a bell curve. There is a much larger pool of people who basically drank bacon grease for breakfast and didn’t live long enough to appear on television. Most people can’t chug bourbon and gravy for a lifetime and expect to become an octogenarian, but the unusually lucky handful who can tend to stand out precisely because they are alive and talking.

. . .

Failure to look for what is missing is a common shortcoming, not just within yourself but also within the institutions that surround you. A commenter at an Internet watering hole for introverts called the INTJForum explained it with this example: when a company performs a survey about job satisfaction the only people who can fill out that survey are people who still work at the company. Everyone who might have quit out of dissatisfaction is no longer around to explain why. Such data mining fails to capture the only thing it is designed to measure, but unless management is aware of survivorship bias things will continue to seem peachy on paper. In finance, this is a common pitfall. The economist Mark Klinedinst explained to me that mutual funds, companies that offer stock portfolios, routinely prune out underperforming investments. “When a mutual fund tells you, ‘The last five years we had 10 percent on average return,’ well, the companies that didn’t have high returns folded or were taken over by companies that were more lucky.” The health of the companies they offer isn’t an indication of the mutual fund’s skill at picking stocks, said Klinedinst, because they’ve deleted failures from their offerings. All you ever see are the successes. That’s true for many, many elements of life. Money experts who made great guesses in the past are considered soothsayers because their counterparts who made equally risky moves that failed nosedived into obscurity and are now no longer playing the game. Whole nations left standing after wars and economic struggles pump fists of nationalism assuming that their good outcomes resulted from wise decisions, but they can never know for sure.

There's much more at the link.  Go read the whole thing - it'll challenge all sorts of preconceived ideas.

This is a truly fascinating article.  I can't recommend it too highly to anyone who has important life decisions to make.  I'm certainly going to apply it to decisions I have to make about my (hopefully burgeoning) career as an author.

Peter