Lies, damn lies, and – or are? – statistics

There are, according to Mark Twain, lies, damn lies, and statistics. And you can, of course, prove just about anything with statistics, 66% of which, according to one wag, are wrong. Even the quote attributed to Mark Twain was apparently said first by British prime minister and novelist Benjamin Disraeli. But Walter Bagehot (the first editor of The Economist), the First Earl of Balfour, Sir Charles Dilke, and others have all been credited with the quotation. Nine out of ten people, though, still say it was Twain.

I’ve never really understood where a lot of statistics come from. When I read that Barack Obama has a less than 50% approval rating, or that 45% of people believe we should pull out of Afghanistan, who’s being asked? I never have been. Ever. And anyway, who cares about vox populi? I’ve always felt that the so-called Man in the Street has no business there, but should be in a library researching his facts before treating us – or the pollsters – to his opinions. And remember, these street sages are some of the same people who believe in flying saucers, or in ‘death panels’, or that the moon landing was actually staged in a Hollywood studio.

According to This Week magazine, 23% of all psychiatrists in the US are in New York City, causing one to wonder whether the mental health of New Yorkers is better than, say, that of the denizens of San Francisco. Then there was the USA Today report that counties with high levels of rainfall had concurrent high levels of autism, despite the fact that autism diagnoses have increased in all climates.

An outfit called STATS (published from George Mason University, USA) honors some of the worst abuses of statistics and science in the previous year in their ‘Dubious Data’ awards. They call out the ‘scorecard on US Health System Performance’ published by the Commonwealth Fund in 2008, which aimed to “measure and monitor health care outcomes, quality, access, efficiency, and equity” in the US. The results were, not surprisingly, dismal and growing steadily worse. Yet, STATS notes that many of the findings came not from patient outcomes but from surveys of patient satisfaction. “In other words, patients with no medical training decided whether the treatment they received was mistaken or unnecessarily repetitious.” Moreover, says STATS, “the scorecard mashed together studies without adjusting for different methodologies, sample sizes and collection techniques.”

So where was the 19^th century French physician Claude Bernard when we needed him? He said that, “the first requirement in using statistics is that the facts treated shall be reduced to comparable units.”

Maybe we’re best off with Andrew Lang, a Scottish scholar (1844-1912) who is quoted in Familiar Medical Quotations thus: “He uses statistics as a drunken man uses lampposts – for support rather than illumination.” Interestingly, Lang was not a medical man but rather a polymath, best known for producing an annual volume of fairy tales.

What all this probably means for would-be statisticians and economists is more rigour in gathering and presenting statistics, and a healthy skepticism in interpreting and using them. Oh, and don’t forget to ask my opinion. Like a stopped clock, I’m right twice a day.

Dr Woods is the former editor of Health Outcomes Communicator.