More on Tamiflu: The Challenges of Getting Good Evidence

I posted some time back on oseltamivir (Tamiflu) and the Cochrane systematic review that concluded that there's no good evidence that this drug prevents serious complications of influenza (despite international public health bodies having spent billions to stockpile it):
http://brodyhooked.blogspot.com/2009/12/bmj-medical-research-is-broken.html

Earlier this year, the BMJ published a further commentary from the Cochrane team, explaining further their difficulties in getting credible evidence to do their review:
http://www.bmj.com/content/344/bmj.d7898

Doshi and colleagues noted that after a good deal of fussing around, they were able to get their hands on portions of the clinical study reports from a number of trials of oseltamivir. They admitted that when this all started, they had no idea of what a clinical study report was. A clinical study report is the province of regulatory agencies; it's what the FDA looks at, for example, when deciding whether to approve a drug for marketing. Academic investigators, by contrast, including the Cochrane folks, generally deal with published study reports in medical journals. The difference can be seen when they noted, "For example, the published version of one cardiac safety trial of 400 patients is seven pages long... compared with 8545 pages for the full clinical study report."

Depite still being unable to get their hands on all the materials from all of the studies, the Cochrane team found temselves the proud possessors of 22,000 pages of documents giving highly detailed information about each study. As a result, "Our new Cochrane review update of oseltamivir engaged the equivalent of two whole time researchers (a junior and a senior) for 14 months."

In short, this account presents a sobering picture of how much we still don't know about the evidence for and against a drug even after reading what purports to be a thorough, well-done systematic review based on published studies. It probably goes without saying that very few academic teams have the resources to let two people spend all their time for 14 months reviewing a single study question.

Since I learned of this paper via the services of Primary Care Medical Abstracts, I also had the advantage of Dr. Jerry Hoffman's trenchant commentary. Jerry asked how likely it was that of the research data that remains unpublished and thus far unanalyzed, any of it supports the wider use of oseltamivir for influenza. If the world were just, there'd be roughly a 50-50 chance that any unpublished data leaned for or against the drug. Based on what we know from past experience, if a study was done and there was any way at all that the results could be spun in a manner that produced positive marketing for Tamiflu, it would probably have taken the drug company (Roche) about 7 nanoseconds to arrange for the data to be published and widely disseminated. So we have good grounds to believe that the unrevealed data is not very friendly to Tamiflu.

Nonetheless, another point that Doshi and colleagues noted in their paper is that the various government agencies cannot see eye to eye on the current data:

"In December 2009, we expressed serious doubts about the credibility of the evidence for oseltamivir because of the inaccessibility of these unpublished trials. Nevertheless, influential organisations such as the US Centers for Disease Control and Prevention (CDC) and European Centre for Disease Prevention and Control continued to cite the Kaiser et al meta-analysis....[a company sponsored study supportive of the drug] Neither agency seems to have done an independent analysis of all available evidence, even after Roche’s public offer to provide full clinical study reports. Their stance is more worrying given that another US agency unambiguously holds the opposite opinion. The FDA, which has reviewed the oseltamivir trial programme in perhaps more detail than anyone outside of Roche, states that “Tamiflu has not been shown to prevent such complications [serious bacterial infections].”... The FDA even sent Roche a warning letter in 2000 instructing it to “immediately cease dissemination of promotional materials” containing “false or misleading” claims, including statements about a reduced risk of influenza complications.... The FDA has, however, not challenged the CDC’s claims."

The Deja Vu Fan Club out there might be commenting at this point that the oseltamivir story seems to be a retread of the reboxetine story: http://brodyhooked.blogspot.com/2010/11/pfizers-reboxetine-latest-in-series-of.html
--in which only one-fourth of the study data on a new antidepressant was published, with that one-fourth favoring the drug, and the 3/4 that remained unpublished showing that the drug was both ineffective and less safe than comparable drugs.