I've been thinking about survivorship bias in data analysis lately, especially when it comes to business success stories. We always hear about the startups that made it big, but what about the thousands that failed?
This seems like one of those statistical illusions in data that really distorts our perception. When we only look at successful cases, we might draw completely wrong conclusions about what leads to success.
Has anyone seen good examples of survivorship bias in action? Maybe in finance, entrepreneurship, or even historical analysis?
Survivorship bias in data analysis is huge in the investment world. All those books about what billionaires do" or "habits of successful investors" - they only study the winners. They never look at all the people who did the exact same things but failed.
I remember reading about a study of mutual fund managers. If you only look at the ones who survived 10 years, they look like geniuses. But if you include all the funds that closed in that period (usually the worst performers), the picture changes completely.
This creates statistical illusions in data that make active management look more successful than it actually is.
World War II aircraft damage analysis is the classic survivorship bias example. The military looked at returning planes and saw more bullet holes in certain areas (wings, tail). They wanted to reinforce those areas.
But a statistician pointed out they were only looking at planes that survived. The planes that didn't return probably had damage in different places - like the engines or cockpit. Those were the areas that actually needed reinforcement.
I use this example when teaching because it's so clear. We tend to focus on what we can see (the survivors) and ignore what we can't see (the failures). This affects everything from product development to career advice.
In tech startups, survivorship bias is everywhere. We hear about the Facebooks and Googles, but not about the thousands of startups that failed doing similar things.
What's particularly insidious is that successful founders often attribute their success to specific strategies or personality traits. But failed founders might have had the same traits and strategies! We just don't hear from them.
This leads to really bad advice being circulated. Work 100 hour weeks!" "Drop out of college!" "Follow your passion!" These might have worked for some survivors, but we have no idea how many people tried the same and failed miserably.