False Claims of Equivalence in the Neurosurgical Trauma Literature: Prevalence and Associated Factors - A 60-year Bibliometric Systematic Review

Introduction: Many neurotrauma studies lack enough statistical power to detect clinically important differences between treatment arms. Challenges include few financial incentives and restricted resources in the highest burden of neurotrauma countries (low/middle-income). This study aims to estimate the prevalence of false claims of equivalence in the neurotrauma literature and identify its predictive factors.

Methods: Negative randomized clinical trials(RCT) enrolling only TBI patients were eligible. The search included:MEDLINE/PubMed; English language; 1960-2020; 15 top ranked journals. False claim of equivalence was defined as insufficient power to detect a clinically meaningful effect: for categorical outcomes, relative difference ≥25% or ≥50%(moderate or large effect, respectively); for continuous outcomes, Cohen’s d ≥0.5 or ≥0.8(moderate or large effect). Each study power was calculated from the number of patients, the effect sizes to be detected, and two-tailed α 0.05 assumption. Standardized differences(STD) between studies with or without false claim of equivalence were calculated and the variables with STD >0.2 or >0.5 were considered weakly and strongly associated with false claims of equivalence.

Results: Fifty-three RCT reviewed, of which 34(64.2%) presented false claims of equivalence:64.2% were unpowered for moderate effect sizes and 26(49.1%) unpowered for even large effect sizes. Most RCT were conducted in high income countries(90.6%) and tested drug interventions(60.4%). Median number of patients was 148 and median follow-up was 6m. Smaller sample sizes and shorter follow-up were associated with false claims of equivalence. Multicenter studies, the presence of a statistician among the authors, a priori sample size calculation and blinding were protective against false claims of equivalence. Studies without false claims of equivalence tended to present lower risk of bias(Cochrane risk of bias tool), mainly in the measurement of outcome domain.

Conclusion : False claims of equivalence are highly prevalent in neurotrauma literature. Multicenter studies, statistician coauthor, sample size calculation and blinding are recommended to improve the quality of neurotrauma trials.