How confidence intervals become confusion intervals

TitleHow confidence intervals become confusion intervals
Publication TypeJournal Article
Year of Publication2013
AuthorsMcCormack J, Vandermeer B, G Allan M
JournalBMC Med Res Methodol
Volume13
Pagination134
Date Published2013
ISSN1471-2288
Abstract

BACKGROUND: Controversies are common in medicine. Some arise when the conclusions of research publications directly contradict each other, creating uncertainty for frontline clinicians.

DISCUSSION: In this paper, we review how researchers can look at very similar data yet have completely different conclusions based purely on an over-reliance of statistical significance and an unclear understanding of confidence intervals. The dogmatic adherence to statistical significant thresholds can lead authors to write dichotomized absolute conclusions while ignoring the broader interpretations of very consistent findings. We describe three examples of controversy around the potential benefit of a medication, a comparison between new medications, and a medication with a potential harm. The examples include the highest levels of evidence, both meta-analyses and randomized controlled trials. We will show how in each case the confidence intervals and point estimates were very similar. The only identifiable differences to account for the contrasting conclusions arise from the serendipitous finding of confidence intervals that either marginally cross or just fail to cross the line of statistical significance.

SUMMARY: These opposing conclusions are false disagreements that create unnecessary clinical uncertainty. We provide helpful recommendations in approaching conflicting conclusions when they are associated with remarkably similar results.

DOI10.1186/1471-2288-13-134
Alternate JournalBMC Med Res Methodol
PubMed ID24172248
PubMed Central IDPMC3818447