Concerns Medical Research Does Not Give Full Story
Concern mounts that medical research does not give the full story
By Diana McCurdy
Be wary next time you read about a medical breakthrough or a miracle drug. New research has found that scientists are disturbingly selective when reporting the results of clinical trials. Many scientists’ cherry-pick favourable results. Others change tack when unexpected or interesting results emerge, breaching research protocols.
A research team led by Oxford University academic An-Wen Chan analysed 102 trials and found that researchers failed to fully report almost two-thirds of the results relating to potentially harmful outcomes.
Half of the results relating to the effectiveness of a treatment were not made fully public.
Many researchers also failed to adhere to basic research protocol.
Scientists are supposed to specify their main objectives - or "primary outcomes" - before beginning a trial to protect against selective reporting.
However, in 62 per cent of cases, researchers changed the primary outcomes in their published reports. In one published trial, the assessment of pain intensity shifted from being a primary outcome to being unspecified. In another, researchers introduced new primary outcomes that were not mentioned in their initial outline.
"The reporting of trial outcomes is not only frequently incomplete, but also biased and inconsistent with protocols," the team reported in the Journal of the American Medical Association. "Published articles, as well as reviews that incorporate them, may therefore be unreliable and overestimate the benefits of intervention."
It's a disturbing prospect. Ultimately, clinical trials influence which drugs your doctor prescribes - and which drugs Pharmac funds.
"The worst possible situation for patients, healthcare professionals and policy-makers occurs when ineffective or harmful interventions are promoted," the team says. "But it is also a problem when expensive therapies, which are thought to be better than cheaper alternatives, are not truly superior."
Many researchers seem oblivious to the potential bias in their published reports. Eighty-six per cent of respondents to the Oxford study denied the existence of unreported outcomes despite clear evidence to the contrary.
Auckland University Clinical Trials Research Unit (CTRU) director Anthony Rodgers describes the team's findings as a "wake-up call", but not surprising.
This study is not the first to cast suspicion upon potential bias in medical research, he says.
In an effort to counter the problem, pressure is mounting for compulsory registration of trials.
In New Zealand, there is no definitive register of clinical trials. Some researchers sign up with off-shore registers but it is not compulsory to do so.
Rodgers is among a group of academics agitating for a compulsory register to be set up in New Zealand. "[It will] basically keep researchers honest and make sure trials get published in the way they were planned and inconvenient things don't get pushed under the carpet."
CTRU research fellow Andrew Jull says registers also help to stop bias from creeping into common medical practice.
When clinicians or government funding agencies want to assess the merits of a drug, they turn to reviews of published trials. Those reviews give an overview of all research into a particular treatment or condition.
But there is a growing concern that non-publication of some trials is skewing results. Australian research has found that trials with a positive outcome are three times more likely to be published than those with a negative outcome, so the public frequently doesn't see unfavourable or inconvenient findings.
"So if your trial finds that something doesn't work or, alternatively, it doesn't find a result full stop, then it is apparently more difficult to get it published," Jull says.
Failure to publish can have dangerous - even lethal - consequences. When a recent review of antidepressant trials looked at published and unpublished research it found that several drugs that seemed beneficial for children actually appeared to cause more harm than good when all published and unpublished research was taken into account.
An even more alarming example occurred in the 1980s and early 1990s when thousands died while using an approved heart attack medication. It was not until 1993 that a trial conducted 13 years earlier was published, along with its evidence of a higher death rate.
Today, failure to publish is increasingly viewed as a form of scientific misconduct. But overcoming journal editors' preference for positive results can be difficult.
Such is the extent of the problem that a journal has been established to cater specifically for negative findings - the Journal of Negative Results in Biomedicine.
Jull says that when journal editors' bias toward positive results is combined with the potentially biased reporting of researchers, it creates a very real danger we will develop a skewed perception of how effective certain drugs are.
Efforts have already been made to conquer the problem. The major medical journals now subscribe to the Consort statement, which provides strict guidelines for unbiased trial reporting. The journal editors have also agreed to try to counter their bias toward positive trials.
More people within the health profession are recognising that they should take into account the results of unpublished trials in their treatment reviews. The Cochrane Collaboration, which was established in 1993 to provide systematic reviews of health research, now uses both published and unpublished data.
Jull believes making it compulsory to register clinical trials would greatly improve public access to results.
"Before you are allowed to proceed, you must register [your trial] with a database that's accessible to all," he says. "So when people reviewing evidence make a decision whether a treatment works or not, they have access to all the evidence and not just the published evidence." More than 500 applications for clinical trials have gone to research ethics committees in New Zealand in the past five years. It is not known how many were later withdrawn and how many have languished unpublished.
Jull says ethics committees have a duty to ensure trial results become public. "If it is unpublished, there is the potential that, even with the best will, prescribers can be misled, and that can have disastrous effects."
At the moment, there is little official interest in setting up a New Zealand register, but Jull believes it would be a simple, cheap exercise to set one up online. Researchers could enter their own details over the internet.
"It's probably easier than in a lot of other countries because we have ethics committees that work to a single operational standard. No clinical trial can proceed without ethics committee approval. It would be relatively simple for the ethics committees to say: Is this registered?"
As well as combating potential bias, a New Zealand register of trials would help scientists to keep tabs on local research, making collaboration easier and preventing duplication, Jull says. Patients and clinicians could also use the register to track down trials in which they could participate.
New Zealand punches above its
weight in performing clinical trials. Jull says it is time
we started contributing to an international register and
ensuring that the results of our research are recognised.