Case Study: Detecting Marketing Bias in Sponsored Research

I would suppose that a newcomer to this debate over Pharma influence, who does not routinely read medical journal articles, would have the following reaction to many of the arguments put forth in previous posts:

Why are you making such a big deal about industry sponsorship of research? It must be fairly easy to read a medical journal article, assess the methods employed, and decide that a study is of good or poor quality--that one can or cannot trust the findings. Isn't that the key variable--that one can or cannot trust the findings--rather than whether or not the study was sponsored by a drug company?

For this reason I feel justified in placing on record a case study, showing just how far one often needs to "drill down" to discover the evidence of bias in a commercially sponsored research publication. For this example I must thank my friends Rick Bukata and Jerry Hoffman and their excellent audio service, "Primary Care Medical Abstracts."

For their May, 2009 issue, Rick and Jerry discussed a paper by Ho et al. in Lancet (a top tier journal, which would lead some to imagine immediately that no possible skullduggery could have made it past the distinguished editors of this journal). The basic idea here is that Merck (who funded the study and whose role is clearly labeled in the paper) has an investigational drug for migraine headache that is in a new class, calcitonin gene-regulated peptide receptor antagonists. Merck obviously hopes that this new medicine, when approved, will quickly become the preferred therapy for migraine in place of the older standbys, the triptan drugs, which are currently going off patent and becoming cheaper.

The study was set up to compare two doses of the new drug, called telcagepant (we assume Merck will think of a melodious brand name instead of this nearly-unpronounceable generic name), the older standard drug, zolmitriptan at maximum recommended dose, and placebo. They looked at a variety of migraine symptoms at both 2 and 24 hours after the initial drug dose.

How did the drugs compare? Here's how the authors of the paper reported their outcomes in the abstract (the only part of the paper that many busy clinicians ever read): "Telcagepant 300 mg is effective as an acute treatment for migraine with efficacy comparable to that of zolmitriptan 5 mg, but with fewer associated adverse effects." In other words the new drug is clearly better than the old drug.

The abstract prepared by the Primary Care Medical Abstracts team expressed the same data but with a more cautious spin: "In this study, a 300mg dose of...telcagepant was not more effective that zolmitriptan for the treatment of acute migraine."

But to get the full story we needed to wait till Jerry Hoffman had his say on the audio disk. Jerry pointed out a feature of the study design that was not mentioned in the abstract at all, and that was only discussed in the methods section and in Figure 1. The study design called for the option for any subject to receive a second dose of medication at 2 hours past the initial dose, if needed. This feature makes sense because migraine headaches often rebound within the first 24 hours or are not completely relieved by the first medication dose, or both. If the subject was assigned to the placebo group or the zolmitriptan group, the second dose would be placebo. If the subject was assigned to either of the telcagepant groups, the second dose was randomized, half the subjects getting a second dose of telcagepant, the other half getting placebo.

Now, stop and think. Can you imagine any legitimate scientific question that would be answered by having the study design as described? I cannot imagine one. I have to conclude that this study design must have been selected purely for its marketing impact, and not as a way of doing science.

So you can see that the playing field was not level--as Jerry pointed out. A single dose of zolmitriptan was being compared to the possibility of repeated doses of telcagepant. (In actual fact, about half of all the subjects required a second dose, so a total of a quarter of the telcagepant subjects got a second dose of an "active" drug instead of placebo.)

This understanding of the true study design puts quite a different spin on the reported outcomes. What if half the zolmitriptan subjects, who needed a second dose, had been given a second dose of zolmitriptan? Would telcagepant still have been "as effective as" zolmitriptan--or would the older standby drug, and the soon to be the probably much cheaper drug, have proven superior?

You can pretty much answer this question for yourself by studying the finer details of Table 3. Efficacy for each study endpoint was expressed as a percentage of subjects who achieved the target outcome. In almost every single category, at 2 hours, the percentages reaching target were higher for zolmitriptan than for either dose of telcagepant, with the single exception for phonophobia (being bothered by loud noises). In photophobia (being bothered by bright light), the higher dose of telcagepant was slightly better than zolmitriptan (51% vs. 50%). For example, a secondary endpoint, "total migraine freedom [at] 2 hr" was achieved by 22.9 % of the higher-dose telcagepant group, and by 27.2% of the zolmitriptan group. The only endpoints for which there was any superiority for telcagepant were those endpoints that looked at sustained effects over 24 hours--where, as we saw, the playing field was not level and a quarter of the subjects in the telcagepant groups got an extra medicine dose. Even there, the differences were nothing to write home about--for "2-24 h sustained pain freedom," 20.2% of the higher dose telcagepant group, vs. 18.2% in the zolmitriptan group.

Exactly how did the methods section of the paper explain the strange methods, by which one active medication group got a second dose that was denied to the other? Here's what Ho et al say: "The efficacy of the optional second dose is not discussed here since the sample sizes were limited. A prespecified meta-analysis across all of the telcagepant phase III studies is planned to address the therapeutic effect of the second dose." Pardon my cynicism but I interpret this statement as: "We won't report the role of the second dose in skewing the study findings now, because then the reader might actually put two and two together. Instead we'll report them (if we ever get around to it at all) separately at a later time, when we hope you'll have forgotten about the issue, and when we can put a favorable spin on whatever we find one way or another." As I have demonstrated, even without an explicit report from the authors, the raw data in the tables, which the average reader would never bother to look at, hint very strongly that the optional dose at 2 hours skewed the results in favor of telcagepant--and without that skew, the expensive new drug would in all probability not have performed as well as the older and cheaper drug.

I know I put you to sleep a long time ago. My take home message is very simple--this is often how carefully you have to read a published article in a medical journal to be sure you have detected any bias that apparently was introduced by commercial sponsorship. And that, in turn, is why commercial bias and conflict of interest is such a plague on scientific medicine.

Ho TW, Ferrari MD, Dodick DW, et al. Efficacy and tolerability of MK-0974 (telcagepant), a new oral antagonist of calcitonin gene-related peptide receptor, compared with zolmitriptan for acute migraine: a randomized, placebo-controlled, parallel-treatment trial. Lancet 372:2115, Dec 20/27, 2008.