It is “common” for researchers to use a different method to analyse clinical trial data in a published paper compared to other locations like trial databases, according to a new review. These discrepancies mean that doctors and public health professionals who try to make decisions based on the best available evidence can face contradictory descriptions of what a clinical trial says.

Scientists typically have a number of ways they can analyse the results of a clinical trial. When submitting a paper to an academic journal for publication, scientists will include only some of the possible analyses of the results of their clinical trial. Many journals now require clinical trials to report their results in a separate database before they can be published, so it is possible to compare the analyses reported in different locations. A number of researchers have written studies comparing the analyses of clinical trials reported in peer-reviewed papers with what was reported in other places.

A group of researchers have published a systematic review of those comparison studies in the open access journal PLOS. The review looked at 22 studies which account for over 3000 clinical trials. These studies looked at whether the analyses and results of clinical trials reported in different places matched. They found that “discrepancies in analyses between publications and other study reports were common.” A discrepancy was when information found in one source was missing from the other or when the two sources contradicted each another.

The Editors’ Summary concludes with how these issues may be addressed:

To make selective reporting of analyses more easily detectable, [the authors] suggest that protocols and analysis plans should be published and that investigators should be required to stick to these plans or explain any discrepancies when they publish their trial results. Together with other initiatives, this approach should help improve the quality of evidence-based medicine and, as a result, the treatment of patients.

Read the full-text of the article, including the Editors’ Summary on PLOS.