"Balance" in the Defense of Poor Arguments Is No Virtue

Some time ago I was driving somewhere with my car radio tuned to the local NPR news station and heard a portion of a discussion regarding one of the services that is supposed to call out falsehoods that appear in media news stories. The discussion immediately turned into a defense of the service against accusations of bias by showing that they had attacked roughly equal numbers of statements by Republicans and Democrats as being false.

My reaction to this weird discussion was: wait a minute, I thought the reason for this service was to tell us what's true and what's false, not to be "balanced" between the two major parties. What if one party told lies 90% of the time and the other one 10%? Is the service then supposed to be "balanced"?

Naomi Oreskes and Erik M. Conway, in their fascinating book Merchants of Doubt, tell us how a small group of scientists has managed to cast doubt on a series of well-established scientific conclusions ranging from tobacco causing cancer to climate change (and what's especially fascinating is that it's many of the same scientists in all these cases). They point out along the way how the "fairness doctrine" in the media works against accurate reporting, since news outlets act as if there's one scientist saying it ain't so while the vast majority of credible scientists in the field say is is so, then they have to give equal time to both viewpoints.

And what does this have to do with the topic of this blog? Be patient, I'm getting there.

A group of three has published a letter to the editor in Nature Biotechnology (subscription required). The three are:

• Roman Lesko, of Navigant Consulting in New York (a firm that does management and economic consulting for various industries, including healthcare systems)



• Samuel Scott of the University of Baltimore Law School



• Dr. Thomas P. Stossel of Harvard and ACRE, whom we last heard from in this blog at: http://brodyhooked.blogspot.com/2012/01/another-planet-heard-from-sunshine-law.html

They start by citing three articles that demonstrate advantages of close associations between the drug and device industries and academic medical centers. The first such article is by our old friend Frank Lichtenberg, the flaws in whose economic research we discussed a while back:
http://brodyhooked.blogspot.com/2008/02/defending-free-lunch-mistakes.html

They then ask whether top-tier medical journals present a fair assessment of the pros and cons of relationships with industry, or whether they rather have an anti-industry bias. They proclaim the latter. They did a study in which three readers, whom they don't name but I would have to guess are the authors, read all articles looking for a variety of characteristics: did they cite evidence; did they address opposing viewpoints, etc. They concluded that the anti-industry articles were seriously inferior in quality to the pro-industry articles--for instance, the former addressed an opposing point of view only 44% of the time while the latter did 100% of the time. They concluded by calling on editors of these journals to present a more balanced picture.

OK, where to start... First of all, most of their measures of the quality of the articles would seem to be quite subjective, and how much trust would one have in the combined judgment of this group of three individuals that they assessed these articles in the same way the thoughtful, average reader would? Second, I don't see how the editors of these journals could be faulted if they were to conclude, for all the reasons repeated ad nauseam in this blog, that there was a much stronger case to be made on the side of skepticism toward industry engagement.

Along the way, the authors repeat the common ACRE lament that there's really no solid evidence that docs' relationships with industry have ever harmed any patients. As I discussed when the article was first published:
http://brodyhooked.blogspot.com/2010/10/major-new-study-published-in-plos.html
--the article by Geoff Spurling and the group from Healthy Skepticism in Australia is probably the best current review of the available facts on this point. If one of the pharmapologist papers ever cited the Spurling study and explained any flaws in the data analysis, then I might credit that gang with actually looking to see what the evidence might be. I have yet to see one of these papers, however, engage or even mention the Spurling study.

The high-tier journals are pretty one-sided right now on the value to physicians and patients of close financial ties with industry. These same journals are also pretty one-sided right now on the value to patients of treating bronchitis with antibiotics. The latter is because the evidence is overwhelming that antibiotics don't make bronchitis better. Maybe the same reasoning applies to the coverage of industry relationships. Sometimes being one-sided can be good.

Lesko R, Scott S, Stossel TP. "Bias in High-Tier Medical Journals Concerning Physician-Academic Relationships with Industry." Nature Biotechnology 30:320-322, April 2012.

ADDENDUM 4/13/12: Either not wishing to pile on the authors of this research letter, or else suffering from momentary brain absence, I did not check out the report of "competing financial interest" for which I had to go back to the on-line version of the above paper. I thought it was interesting enough that I will reprint it here:

Competing financial interests
T.P.S. has been awarded a grant from the Searle Freedom Trust to study physician- and academic-industry relationships. The Trust had no participation in the conception, design, performance or reporting of the study. Although T.P.S. has various industry relationships, the paper does not address any topic relevant to these relationships. R.L. and S.S. received funding from the Association of Clinical Researchers and Educators to participate in the analysis. The Association had no input into the conception, design, performance or reporting of the research described.

So Dr. Stossel has "various industry relationships." (When he gives lectures, he quite proudly fills an entire PowerPoint slide with his list of relationships.) And he has here written an attack on major medical journals for being negative toward physicians who have relationships with industry. But his relationships are not relevant to the topic of this article. Hmmm.