 |
 |
5 November 2009
A study has found evidence of selective outcome reporting in industry-sponsored reports of randomised trials of off-label use of gabapentin. For eight of the twelve reported trials, the primary outcome differed from that described in the protocol. Trials that presented findings that were not significant for the protocol-defined primary outcome in internal documents were either not reported in full or were reported with a changed primary outcome.
Action
Healthcare professionals should be aware of reporting bias. Healthcare professionals should base their prescribing decisions on evidence-based information from organisations with a public sector ethos such as NICE, CKS, SIGN, Cochrane, CRD, Clinical Evidence, DTB and the NPC. These decisions should be based on consideration of the entire body of evidence and not merely rely on the claims made in individual publications or reports.
What is the background to this?
The design of a clinical trial includes specification of primary and secondary outcomes. While it is possible that investigators may find it necessary to change the pre-specified outcomes, or make other changes to the trial protocol once the study has started, any alterations should be highlighted in published reports on the trial and any potential biases should be stated.
This study examined clinical trials of gabapentin for off-label use for migraine prophylaxis, bipolar disorders, neuropathic pain or nociceptive pain, which were funded by Pfizer and Parke-Davis. Outcomes described in published reports were compared with those described in internal company documents. Some but not all of the documents examined in the study were obtained as a result of litigation against Pfizer and Parke-Davis relating to off-label use of gabapentin.
What does this study claim?
The study found that in eight out of twelve published studies, the primary outcome differed from that described in the protocol. A new primary outcome was introduced in six studies; the primary and secondary outcomes were not distinguished in two trials; primary outcomes were relegated to secondary outcomes in two trials; and one or more protocol-defined primary outcomes were not reported in five trials. In addition, trials with findings that were not significant (P>0.05) for the protocol-defined primary outcome, according to internal documents, were either not published in full or were published with a changed primary outcome.
So what?
The study authors conclude that selective reporting of outcomes threatens the validity of evidence for the effectiveness of off-label interventions. They point out that differences between the primary and other outcomes specified in the protocol and those described in the published report are often subtle but may lead to different interpretations of an intervention’s effectiveness, increasing the likelihood that interventions will appear to be effective when they are not. In addition, such biases can lead to the omission of negative findings in systematic reviews of intervention effectiveness and in evidence-based guidelines.
It is important that busy healthcare professionals are aware that claims made in the published literature may be biased and selectively reported. These studies emphasise why clinical decisions should not generally rely on claims made in individual papers or reports, but on evidence-based information from organisations with a public sector ethos such as NICE, CKS, SIGN, Cochrane, CRD, Clinical Evidence, DTB and the NPC. Such trusted sources have robust quality assurance procedures in place to base their advice on all relevant and valid information set in the context of the rest of the evidence.
Further information of sources of bias and the skills required to critically appraise clinical trial reports can be found on the Evidence-informed decision-making 2 section of NPC.
Feedback
Please comment on this blog in the NPC discussion rooms, or using our feedback form.
Make sure you are signed up to NPC Email updates — the free email alerting system that keeps you up to date with the NPC news and outputs relevant to you