This week, movie fans around the country engage in the annual ritual of handicapping the Oscar odds, debating public predictions and seeking signs of how their favorites will fare. Can “Birdman” knock out “Boyhood?” Will the “Selma” snub continue?
Hollywood watchers sometimes try to base their calls on statistics, citing the higher Oscar chances for films already honored with British Academy Film or Screen Actors Guild awards. In 2013, Huffington Post even created an Oscars Predictions Dashboard, based on its own statistical models.
If only mastering statistics were simply another form of entertainment, a parlor game for the Oscars telecast or the NCAA Tournament bracket. But the unfortunate and real life consequences of misunderstanding statistics stretch far beyond a losing office pool. We make poor choices and suffer for our lack of understanding of even basic statistics. Yet our unacceptable level of statistical acumen persists unnoticed. An entire public relations industry pushes the notion of financial literacy and taking personal responsibility for our finances, even as we ignore our inability to grasp even the simplest statistical concepts that determine how we live our lives.
The measles outbreak that counts among its causes a wildly off-the-mark assessment of risk is only the most public of these consequences. Consider two favorite summer pastimes — beach trips vs. going to a baseball game. Fear of shark attacks is always lurking at the beach, and yet the odds of getting killed by a shark are one tenth the odds of getting hit on the head by a fly ball in the baseball stands. Then there’s the recent cholesterol “retraction,” an apparent reversal of three decades of medical advice regarding the kinds of foods we should eat. It’s only the latest health research to draw conflicting headlines over the years and leave us more confused than ever.
The root cause of that confusion isn’t always fraud or scientific error — it’s often a misunderstanding of probability and statistics. A study a few years ago looked at more than four dozen health claims that researchers arrived at by examining existing data for possible associations — not by conducting controlled experiments. These four dozen claims all had one thing in common — they were tested later by controlled experiments. Astonishingly, not one of the claims held up in the controlled experiment.
A key issue here is what statisticians call the “multiple comparisons problem.” Even in completely randomly generated data, interesting patterns appear. If the data are big enough and the search exhaustive enough, the patterns can be very compelling.
That doesn’t mean they are true, or significant. But since we don’t understand the underlying concepts of statistics, like random sampling, we can’t judge for ourselves whether the patterns mean anything. And we can’t base everyday decisions, like what to eat or how much to exercise, with any degree of certainty.
It’s not only our health decisions that take a hit from our statistical illiteracy.
Regardless of our political leanings, most of us would agree that we’d like to see a more stable world and a reduction in global poverty. But an economic historian from British Columbia, Morten Jerven, created a stir last year when he contended economic statistics in many African countries are often suspect. Governments around the world use that data to determine foreign aid, development, credit and other policies, without accurate knowledge of how those countries are really faring.
There’s also an entirely personal aspect to statistics, a growing interest in what scientific research and data can tell us about relationships. It could be useful — but only if we understand whether it’s meaningful. Some recent media attention has highlighted this notion, like linking smaller wedding sizes to shorter marriages. But think of other factors that could be at play, like family disagreements that might mean fewer wedding guests and bigger relationship problems later. Or that lack of money could be a root cause yielding both smaller weddings and relationship troubles. Before expanding your invitation list, it would be more worthwhile to understand how correlation doesn’t necessarily imply anything about causation.
There are some signs that more attention might begin to be paid to our statistical illiteracy problem. STATS.org is a new non-profit, non-partisan organization that describes its aim as analyzing and explaining numbers and statistics in the news, and promoting statistical literacy. As political campaigns increasingly use Big Data and tools such as microtargeting, the public has more direct exposure to statistical concepts, perhaps prompting an interest in them. If you’re an undecided voter, you’ll experience statistical modeling first hand with all those phone calls, texts, and messages.
Recently, a New York Times Modern Love essay on using a scientifically- based survey of 36 questions as a way to establish intimacy and maybe even fall in love went viral, with readers following up with their own stories of how the tool turned out for them. Although things look dire at the moment for our levels of statistical literacy, that doesn’t mean we might not end up embracing the data-driven life at some point. First we’ll have to know how to figure out the things that are real in life, as opposed to the ones simply due to chance.
(Peter Bruce is founder of The Institute for Statistics Education at Statistics.com, the leading online provider of analytics and statistics courses since 2002. He also is the author of the newly-released Introductory Statistics and Analytics: A Resampling Perspective. (Wiley)