A study released yesterday by researchers of the Nordic Cochrane Center suggests that pharmaceutical companies often fail to tell doctors and patients the full story of the negative side effects of the drugs they sell. Comparing seven previously undisclosed clinical study reports (CSRs) from trials of the slimming drug orlistat with publicly available information, the researchers found that data on adverse effects was neither thoroughly gathered nor fully reported, concluding that:
The reporting of trials of orlistat in the 1990s understated harms in the summarised results submitted to the EMA for drug approval and in the published papers. … Reports of these trials seemed to have systematically understated adverse events.
The paper’s authors warn that such “[s]elective reporting of harms can have disastrous consequences”.
Orlistat is a slimming pill commonly sold under the name ‘Alli’ that has been (somewhat controversially) approved for sale without prescription in the US since 2007 and in the UK since 2009. It can be legally bought online in Britain.
Alarmingly, such underreporting of negative side effects appears widespread. The study published today cites previous research showing that “[r]andomised trials generally underreport harms” and that “[i]ndustry-sponsored trials are more likely than other trials to conclude that a drug is safe”.
Oxford scholar and AllTrials co-founder Dr Ben Goldacre agrees that the problem of understated harms goes far beyond one single drug:
We can be absolutely confident that this is a much wider problem than just orlistat. A study by the German government’s cost effectiveness agency looked at the CSRs for all the treatments assessed by that agency over a five year period. They found that CSRs contained more information on adverse events, and treatment effects, than publicly accessible trial reports and journal articles.
The Copenhagen-based team used Freedom of Information legislation to gain access to the CSRs from the European Medicines Agency. CSRs are the documents that provide the most detailed information on clinical trials, but are usually kept out of public view by pharmaceutical companies and government regulators.
The European Medicines Agency has committed to proactively publishing CSRs submitted from 2015 onwards, but refuses to proactively disclose reports it received in earlier years. The clinical trials in question were conducted during the 1990s. In 2007, the European Medicines Agency refused a request by a different group of researchers to access the full orlistat data, leading to a multi-year tug of war over access to trial information.
The European Medicines Agency’s transparency policy remains insufficient, Goldacre argues:
We need access to all CSRs, wherever they have been produced, for all trials on all currently used treatments. It is clearly not enough to share only CSRs on new trials, as the EMA is planning. No doctor in the world prescribes only drugs approved after 2014. We need all the information on all the trials on the treatments we use this year, not the treatments being approved this year!
The Nordic Cochrane Center researchers studied protocol instructions to investigators for reporting harms, the actual reporting of harms in individual CSR records versus summaries, and the final reporting of harms in published papers.
The researchers summarise their findings as follows:
- We found that protocol instructions to trial investigators had the potential to dilute the appearance of drug-associated harms.
- Between 3% and 33% of the total adverse effects from CSR summaries were described in published papers.
- In one trial, we counted adverse events individually and found that both the number of adverse effects and the number of days with adverse effects in participants taking the drug were understated in the corresponding publication.
They conclude that:
In the orlistat trials, we identified important disparities in the reporting of adverse events between protocols, clinical study reports, and published papers. Reports of these trials seemed to have systematically understated adverse events. Based on these findings, systematic reviews of drugs might be improved by including protocols and CSRs in addition to published articles.
If you’re a doctor and rely on medical journals, you may be misled. Doctors don’t have the time to go reading the actual raw data.
The study, titled “Assessment of Adverse Events in Protocols, Clinical Study Reports, and Published Papers of Trials of Orlistat: A Document Analysis”, was written by Jeppe Bennekou Schroll, Elisabeth Penninga, and Peter Gøtzsche. It is published today in the open-access journal PLOS Medicine.
Note: We will shortly publish an interview with study author Jeppe Schroll on our news page.