Six Red Flags for Identifying Falsified Data in Randomized Controlled Trial Publications

Randomized controlled trials (RCTs) are often upheld as the gold standard of medical research. As such, the academic community has been buzzing about news articles from Nature and Science reporting on a research study by John Carlisle. In his study, Carlisle claims that, of 526 evaluated manuscripts reporting RCTs, 73 (14%) had falsified data. Carlisle’s alarming findings have sparked a broader discussion within the academic community about the trustworthiness of published research and the plague of false data in global healthcare research.

Nature, Science, and other outlets primarily focus on systemic causes of and solutions to prevent this problem from occurring in the first place. While these are crucial conversations, these proposed industry shifts and large-scale changes will take time. In the interim, healthcare professionals can take immediate steps to identify and react to falsified data in research. In this article, we’ll discuss some key things that everyone in the healthcare field, from administrators to practitioners, should know about how to identify and react to falsified data in research.

What’s considered ‘falsified data’?

When considering falsified data, the first thing to come to mind is blatant data fabrication. This can involve inventing new information about experiments that were never conducted or modifying the results of performed experiments to better match a study’s hypothesis or produce impressive results.

However, Carlisle’s research suggests that this type of data fabrication is relatively uncommon. Instead, Carlisle explains that most instances of falsified data arise from errors, such as duplicated data entries and typos. While these errors are much less ethically concerning, the unfortunate outcome remains the same: inaccurate data and findings have entered our collective medical knowledge, potentially leading to misinterpretations of results and incorrect conclusions that impact clinical practice.

How can falsified data be identified?

Unfortunately, spotting falsified data can be a highly challenging and time-consuming task. In his research, Carlisle requested anonymized individual participant data for each study and meticulously scrutinized the datasets—most of which spanned hundreds of rows and columns. He checked for suspicious patterns, such as duplicated, missing, or repeated data sequences.

In a similar effort to expose fraudulent clinical trial reports, Mol et al. identified significant data replication and outcome duplication across separate studies published by the same author team. Like Carlisle, they also observed data repetition and unexpected internal consistency between patients’ outcomes and laboratory values, as well as incorrect statistical calculations, such as P values.

For modern healthcare professionals, conducting such in-depth analysis is often impractical. Still, there are some critical warning signs to be vigilant for when reading a scientific research report.

1. Notice when key methods information is missing. Mol et al. found that several allegedly fraudulent articles did not disclose their recruitment sites or exact study time period, making it harder for skeptical readers to verify the study’s legitimacy. Check to see whether critical information that could be used to hold authors accountable for their data is missing.
2. Stay alert for internal inconsistencies. If a research paper reports different overall survival rates in its abstract, results, and figures, it’s worth delving deeper into the details.
3. Check the authors’ history. Researchers with a history of misconduct, undisclosed conflicts of interest, academic scandal, or manuscript retractions may be more likely to produce unreliable results.
4. Trust your instincts. If the results of a study feel too good to be true, be sure to investigate further. Integrity is critical in healthcare research, and raising your concerns to the journal editors is a responsible action.
5. Evaluate the journal. While no journal is infallible in detecting research misconduct, some journals have more rigorous integrity checks than others. Highly respected journals in your field are more likely to invest substantial time and resources in integrity evaluations than smaller journals or those that are dedicated to fast publication turnaround times.
6. Discuss with colleagues. Seek input from peers, whether within your organization or through online communities of healthcare professionals in your specific subject area, to gain other’s perspectives on the paper’s credibility.

The responsibility for safeguarding research integrity extends beyond the authors of a study, journal editors, and peer review team. Healthcare professionals play a vital role in maintaining the credibility and trustworthiness of the field by critically evaluating and responding to research findings. By actively engaging in these practices, you contribute to the overall improvement of research integrity in healthcare.