We’re sometimes asked if it’s still true that around half of clinical trials have never reported results. Yes, it is.
Some people point towards recent studies that found a higher rate of publication than that. Great! With all the campaigning, new laws, new regulations and emerging codes of conduct we and others have worked towards, it comes as no surprise that the situation is gradually improving. But these studies look at clinical trials conducted very recently, often on the newest drugs, and therefore represent a tiny fraction of all the clinical trials that have ever been conducted. There’s a massive backlog of unpublished trials reaching back decades. A large proportion of these past trials, or tens of thousands of new trials, would have to publish results for the percentage of unpublished trials to change significantly.
To suggest that the problem is solved because the situation seems to be improving is to miss the point. We cannot give up those past trials as lost. We need the results of past trials, and urgently, because these are the trials that were done on the treatments that patients use today and will probably continue to use for many years to come.
What’s the evidence for “around half of all clinical trials have never published results”?
The estimate that around 50% of trials have never published results comes from a large systematic review of publication bias funded by the NHS in 2010. (A systematic review pools and assesses all relevant and available data on a specific research question in an unbiased way. Systematic reviews are considered the best evidence. Read more about them here.) In this systematic review, Song et al assessed evidence from hundreds of separate studies on tens of thousands of clinical trials some conducted as far back as the 1950s. Overall, around half of all trials in the studies they examined had never published results, and positive trials were twice as likely to be published as those with negative results. [1]
The systematic review above is currently the best available evidence. Since it was conducted more studies have been published. These include:
Ross et al, 2009, which found that 46% of trials on ClinicalTrials.gov, the world’s largest clinical trials register, had reported results. See the diagram below which breaks down their results by country, trial size, phase and trial funder. [2] [3].
Prayle et al, 2012, which found 22% of clinical trials had reported summary results on ClinicalTrials.gov within one year of the trial’s completion, despite this being a legal requirement of the US’s Food and Drug Administration Amendments Act 2007. [4]
Jones et al, 2013, which found 71% of large randomised clinical trials (those with 500 participants or more) registered on ClinicalTrials.gov had published results. The missing 29% of trials had approximately 250,000 trial participants. [5]
Schmucker et al, 2014, which found that 53% of clinical trials are published in journals. This study analysed 39 previous studies representing more than 20,000 trials. The researchers also found that trials with positive findings were three times more likely to be published than those with negative results. [6]
Munch et al, 2014, which found 46% of all trials on treatments for pain had published results. [7]
Anderson et al, 2015, which found that 13% of 13,000 clinical trials conducted between January 2008 and August 2012 had reported results within 12 months of the end of the trial. By 5 years after the end of the trial, approximately 80% of industry-funded trials and between 42% and 45% of trials funded by government or academic institutions had reported results. [8]
Chang et al, 2015, which found that 49% of clinical trials for high-risk medical devices in heart disease were published in a journal. [9]
These are just some of the recent studies on clinical trial results reporting. With increased attention to and concern about the problem, it is likely that many more will soon be published. Only when all of this recent research is gathered together with all other relevant research and assessed in another systematic review will we know if this new data changes the estimate that around half of clinical trials have ever reported results.
References
[1] Song F, Parekh S, Hooper L, et al. Dissemination and publication of research findings: an updated review of related biases. Health Technology Assessment 2010;14:8:1-93. http://www.journalslibrary.nihr.ac.uk/__data/assets/pdf_file/0005/64751/FullReport-hta14080.pdf
[2] Ross JS, Mulvey GK, Hines EM, et al. Trial Publication after Registration in ClinicalTrials.Gov: A Cross-Sectional Analysis. PLoS Medicine 2009;8;6:e1000144. http://www.plosmedicine.org/article/info:doi%2F10.1371%2Fjournal.pmed.1000144
[3] Chan AW, Song F, Vickers A et al. Increasing value and reducing waste: addressing inaccessible research. The Lancet 2014; 383:9913:257-266. http://www.anzctr.org.au/docs/An-Wen%20Chan_Lancet%20Series’13_reducing%20waste%20inaccessible%20research_2014.pdf#page=3
[4] Prayle AP, Hurley MN, Smyth AR. Compliance with mandatory reporting of clinical trial results on ClinicalTrials.gov: cross sectional study. BMJ 2012; 344:d7373. http://www.bmj.com/content/344/bmj.d7373
[5] Jones CW, Handler L, Crowell KE, et al. Non-publication of large randomized clinical trials: cross sectional analysis. BMJ 2013; 347:f6104. http://www.bmj.com/content/347/bmj.f6104
[6] Schmucker S, Schell LK, Portalupi S, et al. Extent of non-publication in cohorts of studies approved by research ethics committees or included in trial registries. PLOS ONE 2014; DOI:10.1371/journal.pone.0114023.
http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0114023
[7] Munch T, Dufka FL, Greene K, et al. RReACT goes global: perils and pitfalls of constructing a global open-access database of registered analgesic clinical trials and trial results. Pain 2014; 155:7:1313-7. http://www.ncbi.nlm.nih.gov/pubmed/24726925
[8] Anderson ML, Chiswell K, Peterson ED, et al. Compliance with Results Reporting at ClinicalTrials.gov. New England Journal of Medicine 2015; 372:1031-1039. http://www.nejm.org/doi/full/10.1056/NEJMsa1409364?rss=mostViewed
[9] Chang L, Dhruva SS, Chu J, et al. Selective reporting in trials of high risk cardiovascular devices: cross sectional comparison between premarket approval summaries and published reports BMJ 2015;350:h2613 http://www.bmj.com/content/350/bmj.h2613