Know Your Chances by Steven Woloshin, MD, MS, Lisa M. Schwartz, MD, MS, and H. Gilbert Welch, MD, MPH
There are few things that can make you feel dumber than a statistic. That's because there are few things that are more misleading. Suppose one cholesterol drug reduces your risk of dying from a heart attack by 42% and another reduces your risk by only 3.5%. No question about which one you'd take, right? Now suppose they're both the same drug.
That's just one of the many muddles described in the newly released book Know Your Chances: Understanding Health Statistics, by Drs. Steven Woloshin, Lisa Schwartz and H. Gilbert Welch. If you think you're a smart and skeptical reader of health news and pharmaceutical ads, you may want to read this book first and then think again. (See the top 10 medical breakthroughs of the past year.)
The cholesterol numbers are drawn from an advertisement for the statin Zocor — a drug that the authors are careful not to dismiss out of hand, since it can indeed save lives, just not as many as its makers would like you to believe. The ad openly touts the 42% figure, which is based on a study in which 111 out of 2,221 people with heart disease who used Zocor later died of a heart attack. In a control group of heart patients who used a placebo, 189 out of 2,223 died. So the fact is there were indeed 42% fewer deaths among the Zocor users, compared with the controls. But when you consider the absolute risk of death in either group, the results are somewhat less stunning. The risk of death in the placebo group was just 8.5%, compared to 5% in the Zocor group — a difference of a mere 3.5%. Both survival numbers are correct, but which one would you use if you were writing the ad copy?
Slippery statistics like this are everywhere. Take survival rates — which seem so clear but can be anything but. A 2006 study of CT scan screening for lung cancer seemed to produce results that were flat-out dazzling: Of the 174,000 people who are given a lung cancer diagnosis each year, only 5% are still alive 10 years later. But when CT scans were used to detect tumors early, the survival rate leaped to a stunning 90%.
Hard to argue with findings like that, but Woloshin and his co-authors offer a thought experiment: Imagine every single person who contracts Disease X dies of it at age 70. If you receive your diagnosis at age 67, you'll be gone well before the 10-year threshold is reached. Now imagine a new technology that spots early signs of the disease at age 57. If everyone gets the test, the 10-year survival rate soars to 100%, but no one in fact lives a single day longer. As with Zocor, early CT scans and early detection may in fact make a difference in how long someone lives, but in order to determine how big that difference is, you need to peel back the numbers and look more closely at individual cases.
Another, subtler problem can be the difference between what are known as surrogate outcomes and patient outcomes. A new drug or treatment may reliably lower cholesterol, say, or reduce the size of a tumor — these are surrogate outcomes — and the drug-maker would call that a success. But the ultimate goal of treatment isn't simply to give you lab results you can boast about, it's to make you feel better and live longer; those are the patient outcomes. Sometimes though, good surrogate outcomes don't lead to good patient outcomes. Hormone replacement therapy, for example, raises good cholesterol, which helps reduce the risk heart disease. But it also makes the blood more likely to clot, which raises the heart disease risk. A cancer treatment that shrinks the size of a tumor is of limited value if it's soon followed by tumor regrowth.
Another problem — increasingly — is conflict of interest. As pharmaceutical companies fund more of their own trials, the studies may be designed to yield the sunniest results possible. Allowing a new drug to shadow-box against a placebo, for instance, promises more marketable results than pitting it against a competing drug that's already on the market. Publicizing only surrogate outcomes without mentioning whether the patient benefits in any substantive way is another common drug company dodge. So is burying — or at least minimizing — side effects or other shortcomings.
If all of these caveats merely create a whole new layer of confusion for consumers — which they easily could — the authors of Know Your Chances offer two more tools. The first is a simple guide to credible sources of health stats, including the Center for Medical Consumers and Informed Health Online. The second is a pair of simple questions we should all ask ourselves before we make a medical decision: Does the drug or treatment we're considering have any important risks and does it offer a reasonably good chance of doing us real good? A yes to the first and a no to the second is bad news; a no to the first and a yes to the second is good news. As for two yesses or two nos? Alas, even the best book can't make everything easy.