1. The secrecy is all historic – this doesn’t happen anymore
It is still happening. The best available review of evidence from 2010 and the most recent studies on publication of clinical trial results show that between 50% and 98% of trials are publishing results. What could possibly make even 1% secrecy forgivable?
The situation is certainly better than it was. Over 500 organisations globally have committed to the aims of AllTrials. Soon there will be laws in Europe to mandate registration and reporting. The future will be different. But the vast majority of medicines we use every day were approved a decade or more ago. Information on what was found in trials on those medicines is missing, and risks being lost forever to the doctors and regulators who make decisions about medicines unless organisations can be persuaded and pressured to report them.
We have to deal with the legacy of secrecy and get past trials published.
2. The estimation that “around half” of clinical trials have not reported results is wrong
The 50% figure is based on reviews of large numbers of research papers that looked at the situation over a long period of time. In 2010 Song et al produced a very large study of publication bias for the NHS. They looked at hundreds of separate studies of clinical trial publication. Overall, around half of all trials in the studies they examined had never published results, and positive trials were twice as likely to be published as those with negative results[1].
An analysis of trials on the world’s largest register clinicaltrials.gov from 2009 found that 46% of all trials the researchers looked at had never published results[2]. More recently, a 2014 study of all the clinical trials investigating treatments for pain on all 15 of the global national registers that are part of the WHO’s clinical trial register platform found that half of the trials have not published results[3]. This paper from the Lancet in 2014 summarises the situation. Look at Figure 2 for a breakdown of the publication rates of trials from industry and non-industry, from different countries and by the size of the trial[4].
Some recent individual papers have found a higher rate of publication. These studies have looked at small subsets of all trials, generally the most recent trials, from the past couple of years, on the very newest drugs, over short time periods. With all the campaigning, new regulations and emerging codes of conduct, we would hope and expect some of this improvement to be true (although one industry study on missing data also has several methodological flaws). However, all the evidence needs to be integrated before we can know whether there has been an improvement in transparency overall. Furthermore, the most recent trials represent only a very tiny fraction of the evidence that is needed to guide everyday decisions for patients today. Doctors do not practice medicine using only treatments, or trial results, from the past three years. We need all the results, of all trials from the past three decades, and urgently, because these are the trials that cover the treatments that patients use today. Because around half of all trials were not published over many, many years, we will have to uncover a large number of those older trials, for the percentage of all trials published to change significantly.
[1] F Song, S Parekh, L Hooper, YK Loke, J Ryder, AJ Sutton, C Hing, CS Kwok, C Pang, I Harvey. Dissemination and publication of research findings: an updated review of related biases. Health Technology Assessment 2010; Vol. 14: No. 8 http://www.journalslibrary.nihr.ac.uk/__data/assets/pdf_file/0005/64751/FullReport-hta14080.pdf
[2] Ross JS, Mulvey GK, Hines EM, Nissen SE, Krumholz HM. Trial Publication after Registration in ClinicalTrials.Gov: A Cross-Sectional Analysis. Sim I, editor. PLoS Medicine. 2009 Sep 8;6:e1000144. http://www.plosmedicine.org/article/info:doi%2F10.1371%2Fjournal.pmed.1000144
[3] Munch T, Dufka FL, Greene K, Smith SM, Dworkin RH, Rowbotham MC. RReACT goes global: Perils and pitfalls of constructing a global open-access database of registered analgesic clinical trials and trial results. Pain. 2014 Apr 13. pii: S0304-3959(14)00175-4. doi: 10.1016/j.pain.2014.04.007. Online: http://www.ncbi.nlm.nih.gov/pubmed/24726925
[4] Chan AW, Song F, Vickers A, Jefferson T, Dickersin K, Gøtzsche PC, Krumholz HM, Ghersi D, van der Worp HB. Increasing value and reducing waste: addressing inaccessible research. The Lancet 18 January 2014 (Volume 383 Issue 9913 Pages 257-266 DOI: 10.1016/S0140-6736(13)62296-5) Online: http://www.anzctr.org.au/docs/An-Wen%20Chan_Lancet%20Series’13_reducing%20waste%20inaccessible%20research_2014.pdf
3. Regulators see everything
Regulators do not necessarily see everything. In the case of Tamiflu in the UK the manufacturer provided the licensing body MHRA with only 15 out of 74 trials they had conducted. Regulators in many countries do not have legislative power to force trial sponsors to disclose all the information they hold. Furthermore, many trials are conducted on treatments that are already licensed or never put forward to the regulator for licensing.
4. The journals are to blame for refusing to publish negative results
Lots of journals do, including BMJ Open, Trials and the Journal of Negative Results in Biomedicine.
5. We can’t solve this with the current journal publishing system
Publication in a peer reviewed journal is only one way to get information from clinical trials into the public domain, and may not be the best way. The best format for reporting trials, which we should work towards, is a structured report on a publicly-accessible clinical trial register.
NB: The requirement to report was laid down under the Declaration of Helsinki (the World Medical Association’s set of ethical guidelines for medical research using humans) and the 25 items to be reported were set out in the CONSORT (Consolidated Standards of Reporting Trials) checklist. It’s not some new requirement introduced by AllTrials! Research funders and organisations should expect to include the 25 items in any style of report, whether a journal article or clinical study report and the report on the register.
6. This is only a problem with industry supported trials
No. Close to the same percentage of non-industry trials go unreported. Non-industry researchers don’t have the same financial incentives to hold back some of their results (eg getting a new drug licensed), so their failure to report affects a more random mix of positive and negative findings (industry’s non-reporting is 2:1 negative to positive). While the importance of reporting has been vehemently contested in some industry quarters, it has been neglected much more widely beyond industry.
7. Having all the trials out there will just confuse the picture
Well-conducted systematic reviews will take account of trial quality. By contrast missing trials can never be adequately replaced and will always lead to bias in the overall picture.
8. Allowing unqualified people access to information from clinical trials risks them doing inappropriate analysis that will undermine researcher’s work
The answer to inadequate analysis is not secrecy! The culture of secrecy that we’ve had for too long has hardly protected the public from bad science. Moreover, parts of industry have scare-mongered about the risk of unqualified people seeing results. First of all, it’s funny that they don’t mind these unqualified people hearing about the good ones! And if they want to avoid unfounded public scares about drugs, the best way to achieve this is for independent researchers and commentators to correct misinformation. And the best way to achieve this is for them to have access to all the results!
9. Information from clinical trials contains trade secrets and releasing it risks companies’ competitors using it to commercial advantage
We have challenged people to explain what this is and why it can’t be simply redacted and have yet to receive an example. Furthermore, global companies GSK, Boehringer Ingelheim and Johnson & Johnson have all concluded that this is not a problem for them. GSK has said publicly that “the more eyes on our data the better for us.” In Europe, the Ombudsman has declared that information in clinical study reports (which are the reports produced for licensing purposes) do not generally contain commercially confidential information. Usually, companies’ commercially confidential information is protected in patents. Companies get information about each other’s products via patents. Companies are protected from other companies using their data for commercial purposes by something called Regulatory Data Protection for, usually, up to 10 years.
10. All clinical trials should be paid for with public money, private companies should not be allowed to run them
Clinical trials can include thousands of participants and cost millions of pounds to run. In the UK, the NHS has a budget of £8.8 billion for drugs. This is already overstretched. It could not support running trials. The problem isn’t who runs the trials, it’s the lack of transparency and reporting – which is also a problem in publicly funded research.
11. Releasing all trials will expose individual patient data
The AllTrials campaign is not calling for individual patient data to be made publicly available. We expect all clinical trials past and present to be registered and results reported including clinical study reports where these are produced. There are a number of initiatives under way that would allow independent researchers access to anonymised individual patent data from clinical trials without making that data widely publicly available.
12. There are already laws to deal with this
There is no global law on clinical trial reporting. A small number of countries have laws about registering clinical trials carried out in that country; an even smaller number have laws about reporting results from trials; most do not.
Where laws do exist they are not being enforced. In the US, for example, the FDA Amendment Act 2007 requires that clinical trials carried out in the US on currently licensed drugs after 2009 should be registered on clinicaltrials.gov and report results within a year of their end. The law gives the FDA power to fine researchers who do not comply. An independent audit in 2012 found that only 22% of trials complied with the law [5]. Nevertheless no fine has ever been levied against any company or researcher.
The new clinical trials law recently agreed by the European Parliament will require that trials done in Europe are registered before they begin and results are reported within a year. This law will come into effect in 2016 so will apply to trials conducted after that date and will apply to trials on drugs only.
[5] AP Prayle, MN Hurley, AR Smyth, Compliance with mandatory reporting of clinical trial results on ClinicalTrials.gov: cross sectional study. BMJ 2012; 344 doi: http://dx.doi.org/10.1136/bmj.d7373 (Published 3 January 2012)