Two men sitting across from each other at a table filled with papers
Wayne State scientists Frank Cackowsi and Steven Zielske carried out experiments based on a paper they later found to contain false data. Image: Creative Commons, courtesy of Amy Sacka
Legitimate academic journals evaluate papers before publication by having other researchers in the field carefully read them over. But this peer review process is far from perfect. Reviewers volunteer their time, typically assume research is real, and so don’t look for fraud.
Some publishers may try to pick reviewers they deem more likely to accept papers, because rejecting a manuscript can mean losing out on thousands of dollars in publication fees.
Worse, some corrupt scientists form peer review rings. Paper mills may create fake peer reviewers. Others may bribe editors or plant agents on journal editorial boards.
An ‘Absolutely Huge’ Problem
It’s unclear when paper mills began to operate at scale. The earliest suspected paper mill article retracted was published in 2004, according to the Retraction Watch database, which details retractions and is operated by The Center for Scientific Integrity, the parent nonprofit of Retraction Watch.
An analysis of 53,000 papers submitted to six publishers — but not necessarily published — found 2% to 46% suspect submissions across journals. The US publisher Wiley, which has retracted more than 11,300 articles and closed 19 heavily affected journals in its erstwhile Hindawi division, said its new paper mill detection tool flags up to 1 in 7 submissions.
As many as 2% of the several million scientific works published in 2022 were milled, according to Adam Day, who directs Clear Skies, a company in London that develops tools to spot fake papers. Some fields are worse than others: biology and medicine are closer to 3%, and some subfields, such as cancer, may be much larger, Day said.
The paper mill problem is “absolutely huge,” said Sabina Alam, director of Publishing Ethics and Integrity at Taylor & Francis, a major academic publisher. In 2019, none of the 175 ethics cases escalated to her team was about paper mills, Alam said. Ethics cases include submissions and already published papers. “We had almost 4,000 cases” in 2023, she said. “And half of those were paper mills.”
Jennifer Byrne, an Australian scientist who now heads up a research group to improve the reliability of medical research, testified at a July 2022 US House of Representatives hearing that nearly 6% of 12,000 cancer research papers screened had errors that could signal paper mill involvement. Byrne shuttered her cancer research lab in 2017 because genes she had spent two decades researching and writing about became the target of fake papers.
In 2022, Byrne and colleagues, including two of us, found that suspect genetics research, despite not immediately affecting patient care, informs scientists’ work, including clinical trials. But publishers are often slow to retract tainted papers, even when alerted to obvious fraud. We found that 97% of the 712 problematic genetics research articles we identified remained uncorrected.
Potential Solutions
The Cochrane Collaboration has a policy excluding suspect studies from its analyses of medical evidence and is developing a tool to spot problematic medical trials. And publishers have begun to share data and technologies among themselves to combat fraud, including image fraud.
“With more than 50,000 journals on the market, bad papers shopped around long enough eventually find a home.” — Bodo Stern, former editor of the journal Cell
Technology startups are also offering help. The website Argos, launched in September 2024 by Scitility, an alert service based in Sparks, Nevada, allows authors to check collaborators for retractions or misconduct. Morressier, a scientific conference and communications company in Berlin, offers research integrity tools. Paper-checking tools include Signals, by London-based Research Signals, and Clear Skies’ Papermill Alarm.
But Alam acknowledges that the fight against paper mills won’t be won as long as the booming demand for papers remains.
Today’s commercial publishing is part of the problem, Byrne said. Cleaning up the literature is a vast and expensive undertaking. “Either we have to monetize corrections such that publishers are paid for their work, or forget the publishers and do it ourselves,” she said.
There’s a fundamental bias in for-profit publishing: “We pay them for accepting papers,” said Bodo Stern, a former editor of the journal Cell and chief of strategic initiatives at Howard Hughes Medical Institute, a nonprofit research organization and funder in Chevy Chase, Maryland. With more than 50,000 journals on the market, bad papers shopped around long enough eventually find a home, Stern said.
To prevent this, we could stop paying journals for accepting papers and look at them as public utilities that serve a greater good. “We should pay for transparent and rigorous quality-control mechanisms,” he said.
Peer review, meanwhile, “should be recognized as a true scholarly product, just like the original article,” Stern said. And journals should make all peer-review reports publicly available, even for manuscripts they turn down.
Editor’s Note: This article is republished from The Conversation under a Creative Commons license. You can see the original here, and the full length version, also published by The Conversation, here.
Frederik Joelving, Retraction WatchFrederik Joelving is a contributing editor at Retraction Watch. He worked for more than a decade as a health reporter and editor for Reuters. His stories, many of them investigating corruption in science and the health care industry, have also appeared in The New York Times, Science, The BMJ, Slate, VICE News, MedPage Today, and elsewhere. He has an MS in Biology from the University of Copenhagen and an MA in Journalism from New York University.
Cyril LabbéCyril Labbé has a PhD in Computer Science and a MS in Applied Mathematics from University of Grenoble. He is a tenured professor and head of the Information Systems research team (Sigma-team) at the Grenoble Informatics Laboratory, France. His work on text-mining and automatic detection of bogus scientific papers has led to retractions or withdrawals of countless computer science and bio-medical publications.
Guillaume CabanacGuillaume Cabanac is a professor of Computer Science at the University of Toulouse, research chair at the Institut Universitaire de France (IUF), and researcher at the Institut de Recherche en Informatique de Toulouse (IRIT).
The Conversation