Not All Wrong Are Equal
On Regression to the Mean, Correlation Confusion, and the Case for Expected Value
*This is Part 4 in a series on How Not To Be Wrong by Jordan Ellenberg*
In 1933, famed statistician and business mind Horace Secrist published a book that shocked much of corporate America. His book, The Triumph of Mediocrity in Business, explores a concept we all instinctively know to be true, yet often fail to fully grasp: regression to the mean.
Secrist followed the top and bottom performing groups of companies across various sectors of the U.S. economy. What he found was striking: by 1920, the top-performing stores of 1916 had lost much—if not all—of their advantage. They were still above average, but no longer the standouts they once were. Meanwhile, the worst-performing stores had improved considerably. Whatever force had once elevated the top performers or dragged down the bottom dwellers seemed to have faded within six years.
What Secrist was really highlighting was the role of luck in the success we often see. It’s like the story of the star football running back who signs a massive contract and then fails to meet last year’s numbers. Sure, his financial motivation may have changed, or perhaps he’s just not trying as hard. But the more likely explanation is that he simply isn’t as lucky as he was the season before. He’s still a world-class athlete—but delivering record-breaking stats year after year is incredibly rare.
The response to Secrist’s findings was mixed. Some in the business world found it insulting to suggest that superior management and sharp business instincts weren’t the primary drivers of success. Others dismissed the work as obvious, arguing that it was a waste of time to prove something a coin flip could illustrate: even the best heads-flippers trend toward 50% over time.
Yet, despite its simplicity, we still struggle to apply this lesson. We like to believe that chance plays only a small role in our lives, and that our success is purely the result of hard work. While there’s some truth to that, chance is always a factor—and it must be accounted for. Whether things are going well or poorly, they are likely to level out eventually and return to the mean.
"Wrongness is like original sin; we are born to it and it remains always with us, and constant vigilance is necessary if we mean to restrict its sphere of influence over our actions."
Few concepts expose our capacity for wrongness more than correlation. This idea has tripped up even the brightest minds in science and math. We’ve all heard the phrase “correlation does not equal causation,” but the truth runs deeper: correlation is not transitive, and a lack of correlation doesn't mean two things are unrelated.
Consider a real-world example: HDL and heart disease. HDL is widely known as the "good" cholesterol and is associated with lower rates of heart attacks. It seems logical, then, that anything that raises HDL would also reduce heart attack risk. Enter niacin—a drug proven to raise HDL levels. But despite that, studies show niacin doesn’t reduce heart attacks at all.
So what went wrong? We made a faulty assumption: that correlation is transitive. Because niacin increases HDL, and HDL is associated with fewer heart attacks, we assumed niacin would reduce heart attacks. But it doesn’t. Just because A is correlated with B, and B with C, doesn’t mean A is correlated with C.
It’s easy to fall into this trap because it feels logically airtight. If I live in the same town as Bill, and Bill lives in the same town as Bob, then logically, I must also live in the same town as Bob. That’s transitivity. But correlation isn’t logic—it’s messier. The systems we’re trying to affect, especially in the human body, are incredibly complex. Even small changes can ripple out in unpredictable ways.
So how can we, in good conscience, ever make scientific recommendations? That brings us back to the concept of expected value explored in part 3 of this series. We may never know the exact expected value of an action, but we aim to be reasonably confident that it’s positive. Take smoking and cancer. We still don’t have a single definitive study that proves smoking causes cancer in every case—but the expected value of discouraging smoking is clear. Even accounting for uncertainty, the calculation comes out on the side of saving lives.
We try to strike a balance: reducing risk, increasing life expectancy, and making hard trade-offs—like taking away a cigarette someone might enjoy. We may not be 100% certain in every case, and we may still be wrong. But we can try to be wrong in the right direction—one that saves lives.
Not all wrongs are created equal.