Depressingly Low Scientific Standards

By Michael Fumento

American Outlook, Fall 2002
Copyright 2002 Hudson Institute

  Print this  Print this    Make text larger    Make text smaller

Few drugs inspire more animosity among people who don’t use them than Prozac and its antidepressant cousins. On the one hand, they’re derisively described as "happy pills," capable of slapping a smiley face on anyone. Actually, decades of research have shown that only those suffering true clinical depression benefit from them. Even then, the pills merely bring patients up to the level of non-depressed persons.

Anti-depressants have been miracle drugs for the mentally ill, but detractors alternatively call them "happy pills" or "worthless."

A second, contradictory claim is that the pills don’t work at all, except psychosomatically. Every so often a researcher releases an analysis of clinical drug trials that purportedly shows that antidepressants are little more effective than placebos. It is little wonder that these analyses always reach the same conclusion, because they always use the same methodology and the same data source. These reports are not only as baseless as the "happy pill" attacks; they’re downright dangerous because they can encourage depressed people to quit taking their medicine.

The latest such analysis appeared recently in the American Psychological Association journal, Prevention and Treatment (July 15, 2002). Chiefly authored by University of Connecticut psychologist Irving Kirsch, it combined clinical trials of six different antidepressants. Kirsch’s conclusion was that "80 percent of the response to medication was duplicated in placebo control groups." Given the potential side-effects of antidepressants, Kirsch opined, "Medication might best be considered a last resort." Several pundits then pumped out op-eds claiming that this same conclusion had been reached in an article by psychiatrist Arif Khan of the Northwest Clinical Research Center in Bellevue, Washington — notwithstanding that neither the article nor even a promotional press release about it had appeared at the time.

To turn to what has been published, however, Kirsch’s analysis is so filled with problems that it’s downright depressing in itself. One Godzilla-sized clue that something is terribly wrong with his conclusion is that the trials he lumped together, when looked at individually, show that the various antidepressants themselves are fairly consistent with each other in how well they treat various facets of depression. What changes from study to study is the efficacy of the placebo.

How can this be?

The difference between the drug and the placebo is greatly determined by several factors, including the intensity of the depression, the duration of the illness, and even the length of the clinical trial itself.

"At the milder end of depression, that without mania or mood-swings, you get about a 40 percent response rate [signifying effectiveness] for placebos," notes Steven Dubovsky, professor of psychiatry at the University of Colorado Medical School, Denver. "But if you look at psychotic depression, the response rate is zero. For other severe forms and for chronic depression, the response rate is also very low," he told me. Dr. Khan himself, in a commentary in the February 2002 Journal of Clinical Pharmacology, stated this: "The frequency of statistically significant differences between antidepressants and placebos was higher in the trials that included patients with more severe depression."

This completely refutes Kirsch’s conclusion, because the worst-off patients are generally kept out of clinical trials. "You cannot participate if you’re suicidal, because people think it’s unethical to give placebos to people who might kill themselves," explains Dr. Dubovsky. "They also usually exclude anybody with concomitant illness, extreme depression, and chronic depression."

Also exaggerating the apparent placebo effect is the fact that the studies define "response rates" as an improvement of at least 50 percent. "But who goes to the doctor for a 50 percent improvement?" asks Dr. Dubovsky. The higher the threshold for defining "improvement," the more effective the antidepressants look.

Placebos may also appear in some studies as effective as antidepressants or even more so because the power of the placebo can be immediate, whereas antidepressants take weeks or months to begin working. "The older studies lasted only four weeks," Dr. Dubovsky notes. "Even the newer ones will go only six to eight weeks. Yet there’s good recent data showing that some patients may take up to three months to get a full response, especially with severe depression." Short studies don’t give the real drugs time to kick in, and they don’t allow for the effect of placebos to fade away. Yet Kirsch’s analysis and others that have reached similar conclusions lump all the outcomes together, regardless of study length.

The clinical trials also showed antidepressants to be clearly superior in preventing relapses. According to Dr. Walter Brown, clinical professor of psychiatry at Brown University Medical School in Providence, Rhode Island, "Among patients who have improved with an antidepressant and are then assigned to either placebo or drug continuation, it is not uncommon for those assigned to drug to have a relapse rate of about 10 percent over a year but those assigned to placebo to have close to a 50 percent relapse."

Finally, the duration of the depression doesn’t influence the response to antidepressants, but it does to placebos. "Placebo response rates as low as 13 percent have been reported in patients depressed for two years or more, with drug response rates in the same patients close to 60 percent," according to Brown. With antidepressant drug trials, insufficient attention to these details makes for lousy science. When the subject is a debilitating and costly disease that often ends in suicide, it can be the scientific equivalent of involuntary manslaughter.


Read Michael Fumento’s additional work on statistics.