 |
 |
18th August 2009
A recent study identified how citations made in the biomedical literature may selectively report work that does not represent the spectrum of available evidence, and can be used to establish authority for unfounded scientific claims.
Level of evidence
Level 3 (other evidence) according to the SORT criteria.
Action
Healthcare professionals need to aware of these newly reported forms of bias which involve the inappropriate use of citations in medical literature to support scientific claims. Citations can be used to give a distorted view of the claims they purport to represent. Healthcare professionals should base their prescribing decisions on evidence-based reviews of studies from public sector funded, trusted sources of information, e.g. NICE, Cochrane, CKS, NPC. These decisions should be based on consideration of the entire body of evidence and not merely rely on the claims made in publications of clinical trials.
What is the background to this?
Citations are routinely used to support claims made in the scientific and medical literature. They aim to give credibility to views made on the basis of these claims. They are an essential aspect of the critical appraisal and review of medical literature; however, inappropriate use of citations for clinical claims may not necessarily provide a view that is representative of the entire body of clinical evidence and may promote a biased and unfounded view. A recent detailed analysis of the biomedical literature was carried out to investigate these issues using an example of a widely held scientific belief.
What did the study find?
By constructing a detailed citation network, linking 242 articles (675 citations) addressing the belief (i.e. that β-amyloid is produced by and injures skeletal muscle fibres in the muscle disease sporadic inclusion body myositis), the authors identified how citations were frequently used in a distorted manner for persuasion and used to establish unfounded scientific claims as fact. Distortion in the use of citations included the following:
- Citation bias: Systematic ignoring of papers that contain content conflicting with a claim.
- Bolster claim: justifying animal models to provide opportunities to amplify claim
- Amplification: Expansion of a belief system without data. Citation made to papers that don’t contain primary data, increasing the number of citations supporting the claim without presenting data addressing it
- Invention:
- Citation diversion — citing content but claiming it has a different meaning, thereby diverting its implications
- Citation transmutation — the conversion of hypothesis into fact through the act of citation alone
- Back door invention — repeated misrepresentation of abstracts as peer reviewed papers to fool readers into believing that claims are based on peer reviewed published methods and data
- Dead end citation — support of a claim with citation to papers that do not contain content addressing the claim
- Title invention — reporting of “experimental results” in a paper’s title, even though the paper does not report the performance or results of any such experiments.
How does this relate to other studies?
The authors identified the following as being already known about the topic:
- In addition to its scholarly use, citation has social uses, both self serving and as a tool for persuasion
- One distortion of this persuasive aspect of citation, citation bias, has been recognised in clinical trial reporting where it may lead to false belief about a therapy’s efficacy.
Information on evaluating the evidence from clinical trials can be found on the Evidence-informed decision-making section of NPC.
So what?
This study only uses one example of a claim made from the literature for its analysis. However, it raises general concerns about the inappropriate use of citations to support statements in an attempt to persuade people of a biased view that may not be factually based. As pointed out in the rapid responses to this article, it also raises important concerns about the efficacy of peer review and journal editing in maintaining the quality of published material.
An editorial accompanying the article points out that a clear message from the study is that investigators have an obligation to critically appraise existing evidence and develop their own interpretation of the results of individual studies. It also point out the fundamental need for systematic reviews in all types of research and states: “By selectively citing studies or unsystematic reviews that suit a particular hypothesis, a bias so serious that the recommendations put forth are erroneous can be introduced. We can no longer continue to choose and selectively report work that does not represent the spectrum of available evidence.”
Busy health professionals should be aware of information from sources such as narrative reviews that may use selected citations to give credibility to views expressed. These may not represent a balanced view of the entire literature available on the matter. It is too much to expect busy health professionals to investigate the validity of claims made in all the literature to which they are exposed. However, it is important that they are wary of the fact that claims made in the literature may be supported by biased citations that distort the facts. This study emphasises the need for healthcare professionals to base their decision making on evidence-based reviews of studies from public sector funded, trusted sources of information, such as those provided by NICE, Cochrane, CKS, and NPC.
Study details
Greenberg SA. How citation distortions create unfounded authority: analysis of a citation network. BMJ 2009;339:b2680
Design
A complete citation network was constructed from all PubMed indexed English literature papers addressing the belief that β-amyloid, a protein accumulated in the brain in Alzheimer’s disease, is produced by and injures skeletal muscle of patients with inclusion body myositis. Social network theory and graph theory were used to analyse this network.
Results
242 articles (containing 675 the citations) were identified containing statements addressing the belief described above. The article contains many examples of how citation may be used for self-serving purposes or as a tool for persuasion. Of the ten most authoritative papers, four provided experimental data addressing the claims, all four from the same laboratory, and two of which probably reported mostly the same data without citing each other. Major technical weaknesses were present in these papers. In contrast, six primary papers that contained data that refuted or weakened the claims received no or few citations. Whereas, the supported papers received 94% of the 214 citations to these primary data, the six papers containing data that weakened or refuted the claim received only 6% of these citations. Please see the full article for other examples of citation bias, as well as citation diversion, amplification and invention. The authors also identified how elements of bias, amplification, and invention were used indirectly to support many requests for research funding.
Sponsorship
The author is in part supported by grants from the National Institutes of Health and the Muscular Dystrophy Association. These grants did not contain specific aims directly encompassing this research.
Feedback
Please comment on this blog in the NPC discussion rooms, or using our feedback form.
Make sure you are signed up to NPC Email updates — the free email alerting system that keeps you up to date with the NPC news and outputs relevant to you.