MedScape Article –> To Maintain Trust in Science, Lose the Peer Review

To Maintain Trust in Science,
Lose the Peer Review

Benjamin Mazer, MD, MBA; John M. Mandrola, MD

February 19, 2019

 

“Trust me, I’m a doctor.” In the Internet age, that phrase has never been more fraught. Uncertainty is the rule in medicine and science.

In an ideal world, doctors, scientists, and laypeople would independently evaluate the evidence behind any scientific or medical statement.

Instead, scientific and medical evidence increasingly comes from small groups of sub-specialists who write in obscure prose for academic publications that few can access. That leaves the media to disseminate findings from thinly read scientific journal articles that are held up as totems of authority.

Editors-in-Chief Blame the Media

But there’s a hitch. The Internet’s democratization of all voices allows misinformation—unintentional or malicious—to easily spread. Experts are scared. Recently, the editors-in-chief of the world’s top cardiology journals issued a joint statement warning about the spread of medical misinformation.^[1]

Their examples were patient distrust in statins and vaccines. These editors warn us that “individuals who are neither physicians nor scientists, but often with a specific agenda, have outsized influence over our lives.” They exhort the media to “do a better job.” The editors claim that they carefully evaluate each piece of information their journals publish because “lives are at stake.”

BUT {my editorial insertion!}

In truth, scientific journals have always been a weak check on misinformation. For every journal article published, only a few volunteer reviewers evaluate the work before publication.

The vulnerabilities of “peer review” have been described even by those charged with managing it, such as former BMJ editor Richard Smith.^[2] He and his colleagues have intentionally introduced errors into manuscripts sent for peer review, and found that these deliberate errors were often missed.^[3]

Reviewers vary greatly in how closely they examine a manuscript; they often have their own agenda and conflicts biasing their evaluation; if flaws are identified after publication, the study is almost never retracted.^[4]

Social Media Didn’t Create Anti-vaxxers

Using distrust in statins and vaccines as examples undermines the editors’ point. The modern anti-vaccine movement was not started on Facebook or Twitter, but by a peer-reviewed study published in the Lancet, one of the world’s leading medical journals. Even as the harm of this fraudulent study became clear, it took more than a decade for the journal to retract it.

Also, equating vaccine denialism and statin hesitancy hurts their case, because the choice to take a statin represents a near-perfect example of a decision that is sensitive to patient preferences.

Vulnerabilities in our scientific publishing system have not gone unnoticed by malign actors. For-profit “predatory journals” allow anyone to garner the authority of a scientific publication for a fee, often bypassing peer review. Individuals with fringe agenda even create their own journals to give an air of credibility to antiscientific views, such as creationism or climate change denial.

Peer Review Reimagined

Is the solution to “fake news” in science and medicine for social media to emulate journals? We don’t think so. Traditional scientific checks and balances don’t always work, and they can’t keep up with the pace of social media. A prominent editor and researcher recently wrote that “journals as we have known them are approaching their final act.”^[5]

To bolster trust, radical changes are needed in the way we share evidence-based information with each other and the public.

Authorities argue that peer review is “the worst way to judge research, except for all the others.” Ironically, this is not an evidence-based view. A Cochrane systematic review of 28 studies concluded that “little empirical evidence is available to support the use of editorial peer review as a mechanism to ensure quality of biomedical research.“^[6]

Peer review has never been rigorously pitted against review by other experts (editor’s Note: i.e., including midwives when assessing labor and birth practices) or against less restrictive methods of online publication.

Peer review is premised on the notion that scientists working in the same discipline are best-equipped to judge a study. To be sure, subject expertise is an indispensable part of evaluating evidence. But other working scientists are not the only audience for journal articles. Once published, a study can lead to a cascade of mainstream media reports, press releases, social media discussion, and political consequences. Today’s digital networks rapidly spread truth and misinformation alike. Most scientists are not equipped to evaluate these potentially global consequences.

Beyond Scientists

This raises the question of who is qualified to review a study. We believe no single profession can evaluate the entirety of the context and consequences of science.

We propose that people with expertise in journalism and public policy could help improve translation of science.

Experts in these disciplines are trained to evaluate the entire context of an event, to seek multiple viewpoints, and to write in a fair and accessible manner.

Because they are better-equipped to tease out fraud and bias, and to consider the downstream consequences for the public, people with such expertise should be regularly reviewing and even editing scientific manuscripts. In close collaboration, scientists, journalists, and policy experts could be even more effective.

It will not be cheap to reorient the scientific literature toward a journalistic, context-aware approach, but the money is there

The for-profit scientific publishing industry has annual revenues over $20 billion, some of the highest margins of any industry.

At the same time, the decline of traditional news media has created a glut of underemployed journalists.

 

This new interdisciplinary “contextual review” could be reserved for pivotal studies that will influence patient care or reframe our understanding of fundamental scientific models; niche, technical work can be evaluated in real time on more affordable preprint servers by the scientists who will make use of it.

Scholarly publications are no longer solely a means of communication between experts. They receive immediate attention by the media, patients, and policymakers.

To restore trust in science and to better communicate it to a global audience, we must let go of peer review.

 

Follow Medscape on Facebook, Twitter, Instagram, and YouTube

Medscape © 2019 WebMD, LLC

Any views expressed above are the author’s own and do not necessarily reflect the views of WebMD or Medscape.