Wednesday, November 18, 2009

No Fair Peeking? More Questions about Early Stopping of Trials

In the previous post I mentioned the recent findings about ezetimibe, which happened to be part of the study called ARBITER 6. As one evidence-based blogger, Dr. David Rind has noted--
http://www.evidenceinmedicine.org/2009/11/arbiter-6-and-combining-therapies-with-statins.html
--the results as published could be consistent with three different possibilities:
  1. Ezetimibe adds nothing to heart disease prevention; niacin is helpful
  2. Niacin is slightly helpful or neutral and ezetimibe actually worsens arterial plaque
  3. All the results of the study are due to chance alone
Option #3 becomes meaningful when we note that the actual hard outcomes in the study were 2 bad events in the niacin arm and 9 bad events in the ezetimibe arm. That is, move just a couple of bad events to a different column and the whole outcome changes.

If the study had been continued longer and more hard outcomes accumulated, we'd know a good deal more about which option made the most sense. (And, of course, knowing which means a lot in terms of what treatments docs should recommend to patients.) So it was therefore very interesting that the study was stopped early. The authors state that this was due to a predetermined efficacy endpoint having been reached:
http://content.nejm.org/cgi/content/full/NEJMoa0907569

In theory, a data safety and monitoring board is supposed to be completely independent of the investigators and the commercial sponsor of a study. In actuality, in the past (as I have noted in other posts a while back), some decisions of the "independent" boards have been eerily like the decisions that a commercial sponsor would have made had the goal been to maximize the marketing advantage of the study outcomes. In this case a quick stoppage of the study may have favored the extended-release niacin product being promoted by sponsor Abbott Labs. And a stoppage due to efficacy is stranger than stoppage due to safety concerns, especially when the main outcome measure--thickening of the wall of an artery--is a surrogate marker rather than an actual disease outcome, as Dr. Rind notes.

I remain ignorant of the exact means by which a data monitoring board might be secretly influenced by commercial marketing pressures. But if a further hint is needed that somebody just may be sneaking a peek where they are not supposed to, ARBITER 6 may provide the hint.

(Hat tip to Marilyn Mann for Dr. Rind's blog.)

6 comments:

Anonymous said...

For someone who writes a blog subtitled "Ethics, Medicine, and Pharma" you really have a very limited understanding of the ethics of clinical trials.

In this study, significantly greater reductions in CIMT were observed in the niacin group compared with the ezetimibe group. Consistent with that, major cardiovascular events occurred more frequently in the ezetimibe group. Both effects were statistically significant.

The rate of major CV events in the ezetimibe group was not low (as you imply). Nine of 165 (5%) had events.

The data indicate strongly that ezetimibe was inferior to niacin. The most likely outcome of continuing the trial would be that patients in the ezetimibe group would continue to experience major CV events at a significantly greater rate than patients in the niacin group.

Can you explain how continuing the trial under such circumstances could be consistent with principles of the Declaration of Helsinki?

To put the question another way, would you consider ethical to start a similar trial now that the results of ARBITER-6 are known?

Moreover, you imply that the DSMB may have stopped the trial to favor Abbott. Can you explain how this might have occurred given that the paper states that the DSMB "evaluated the end-point data without knowledge of the treatment assignments"?

Unknown said...

Trials stopped early because of efficacy, should do so when the DMC considers that the accumulated data is sufficient for decision makers to make an informed choice. The data were there to recommend niacin for CV risk reduction before this trial was conducted. The data were not there for ezetimibe. Do the data from this trial conclusively show that ezetimibe is harmful or not beneficial compared to niacin? No, because the small number of events and the fact that the trial was stopped early itself biases the results upward towards overestimation (see Montori et al, JAMA 2005). How do we know these were not sufficient data to justify the immediate cessation of exposure to ezetimibe in the control group? The continued use and prescription of ezetimibe and Vytorin to patients in the community despite these results indicates that the accumulated data has not convinced clinicians that there is a problem with this drug. This justifies continuing with a trial that makes sense. Why this trial was conducted in the first place remains a mistery, however. Ezetimibe needs to be tested against statins in a large secondary prevention trial; before this is completed, this drug should be considered experimental.

Anonymous said...

“Do the data from this trial conclusively show that ezetimibe is harmful or not beneficial compared to niacin? No, because the small number of events and the fact that the trial was stopped early itself biases the results upward towards overestimation (see Montori et al, JAMA 2005).”

You fail to distinguish between the ability to accurately determine the size of an effect and the ability to determine whether an effect exists at all. It is true that small numbers of events and early stoppage of a trial (resulting in fewer events overall than initially anticipated) can reduce our ability to accurately assess the *size* of an effect (that is what Montori et al showed). In the example at hand, it is not possible to say with great confidence that the rate of major CV events is 5 times greater with ezetimibe than with niacin.

However, you are wrong to say that the trial did not conclusively show ezetimibe to be inferior to niacin. The evidence was clear: in this patient population there were statistically significant differences in CIMT and the rate of major CV events in the two trial arms. We cannot be sure of the size of the effect but an effect exists. This is what the statistical analyses tell us. To simply ignore them is not consistent with either a scientific approach or with an evidence-based medicine approach.

“How do we know these were not sufficient data to justify the immediate cessation of exposure to ezetimibe in the control group? The continued use and prescription of ezetimibe and Vytorin to patients in the community despite these results indicates that the accumulated data has not convinced clinicians that there is a problem with this drug.”

The trial should have been continued because its unpublished results had not affected clinical practice? I think that’s a tautology, isn’t it?

“Trials stopped early because of efficacy, should do so when the DMC considers that the accumulated data is sufficient for decision makers to make an informed choice.”

This statement exhibits an extraordinary lack of concern for the health and wellbeing of trial participants. You are suggesting that the trial should have continued even though the (by far) most likely outcome was that patients randomized to ezetimibe would experience major CV events at a greater rate than those randomized to niacin.

It’s important here to consider what constituted a major CV event in this trial: myocardial infarction, myocardial revascularization, admission to the hospital for an acute coronary syndrome, or death from coronary heart disease.

Each of those events represents a substantial risk to patients. You suggest that the trial be continued and ezetimibe patients be exposed to an increased risk of these events so that “decision makers” have enough data to make an “informed choice”. No indication of what such a threshold might be (other than it being more than achievement of the primary and secondary endpoints, one supposes). And no apparent consideration at all for the rights of the study participants.

This would be a good place to review some aspects of the Declaration of Helsinki:

B11. It is the duty of physicians who participate in medical research to protect the life, health, dignity, integrity, right to self-determination, privacy, and confidentiality of personal information of research subjects.

One also wonders how you suggest handling informed consent in this situation. Should newly enrolled patients have been informed of the results of the interim analysis? Or should they have been kept secret from them? What about patients already randomized to treatment? Again, it might be time to review the Declaration of Helsinki:

B24. In medical research involving competent human subjects, each potential subject must be adequately informed of the aims, methods, sources of funding, any possible conflicts of interest, institutional affiliations of the researcher, the anticipated benefits and potential risks of the study and the discomfort it may entail, and any other relevant aspects of the study.

Anonymous said...

Dr. Brody, your failure to respond to the queries above is telling.

Howard Brody said...

Anonymous questions why I have not replied to the two critical comments left on this post. I did not reply because it seemed to me that the most pertinent defense of what I said in the post, can be found in the blog post by Dr. Rind that I was commenting on and referring the reader to. If you will read Dr. Rind's initial post you'll see why he thinks that the early stoppage of the trial was NOT justified by the factors Anonymous lists. Dr. Rind's discussion seemed plausible to me, but I also wanted to be sure that readers had access to Anonymous's contrary viewpoint, for which I thank him or her.

Anonymous said...

Actually, Dr. Rind does not state that closing the trial early was not justified, you do. You're also the one who impugns the motives of the trial investigators. And you are also the one who puts himself forward as an ethical expert. That's why I posted on your blog, not Dr. Rind's.

It is, of course, your choice as to whether you respond to the questions above. However, if you choose not to defend your opinions you must understand that most will reach the same conclusion I have: they are indefensible.