FORTY PERCENT OF TRIALS ARE OMITTED FROM REHABILITATION META-ANALYSES DUE TO MISSING EVIDENCE

S.G. Lazzarini¹, M. Stella Yousif², S. Bargeri³, G. Castellini³, S. Gianola³

¹IRCCS Fondazione Don Carlo Gnocchi, Milan, Italy, ²University of Rome Tor Vergata, Department of Clinical Science and Translation Medicine, Faculty of Medicine and Surgery, Rome, Italy, ³IRCCS Istituto Ortopedico Galeazzi, Unit of Clinical Epidemiology, Milan, Italy

Background: The current best evidence is usually informed by systematic reviews (SRs) of randomized controlled trials (RCTs). The pooled treatment effect estimates of meta-analyses can be biased when RCTs are omitted from the meta-analyses due to outcome-related reasons (i.e. missing evidence).

Purpose: We aimed to assess the prevalence of RCTs omitted from meta-analyses of SRs in the rehabilitation field and the reasons for missing evidence. Secondly, we aimed to compare the prevalence of RCTs omitted from Cochrane (CSRs) and from non-Cochrane Systematic Reviews (nCSRs).

Methods: This is a cross-sectional meta-research study, prospectively registered (https://osf.io/p25zy/). We started from a sample of 827 SRs published in 2020 identified by Gianola et al. We defined the index meta-analysis (IMA) as the meta-analysis of the primary outcome of the main comparison or, if not clearly identified, the first one reported in the results section. Starting from all the eligible studies for the chosen comparison, we identified all RCTs omitted (i.e., not included) from the IMA, either from the list of the included studies of the SRs or, when available, from the list of studies excluded because of "no relevant outcome data" or similar reasons. The reasons for RCTs' omission from the IMA were assessed using and adapting the classification of Yordanov et al.

Results: 131 IMAs comprising a total of 1761 eligible RCTs were selected. Overall, 100 IMAs omitted evidence, 16 IMAs included all eligible studies, whereas the remaining 15 IMAs reported the exclusion of RCTs for outcome-related reasons but we were not able to assess them due to bibliographic references not reported. Out of 1761 total eligible studies, 717 RCTs (40,7%) were omitted from 100 IMAs: for 38,8% the reason was not clear due to lack of a registered protocol (i.e., "Unable to distinguish between selective reporting and inadequate planning"), 17% due to "inadequate or alternative or missing planning", 15,1% were justified not to be included, 8,4% due to "incomplete reporting", 5,2% due to other situations, 3,3% due to "selective reporting", and 11,7% were not assessed being non-English trials or trials with full text not available. Comparing CSRs and nCSRs, the former presents a higher prevalence of missing evidence (59,2% vs. 35,4%, respectively).

Conclusions: Outcome non-reporting bias (i.e., "selective reporting" and "incomplete reporting") and lack of guidance on the outcomes to be measured as a minimum (i.e. "inadequate or alternative or missing planning") represent important reasons for missing evidence. Similarly, lack of trial registration prevents the identification of the reason for missing evidence. Consequently, better reporting, prospective protocol registration, and the definition of core outcome sets for every clinical condition are needed to avoid research waste and improve conducting of RCTs in the rehabilitation field.

Implications: To the best of our knowledge, this is the first study in the rehabilitation field to focus on missing evidence from SRs and investigate the reasons behind this. The work highlights some issues in rehabilitation research, implying that meta-analyses’ results might underestimate or overestimate the interventions’ effects deviating from the true effect estimates, thus we suggested possible solutions to such issues.

Funding acknowledgements: This study was supported and funded by the Italian Ministry of Health - Ricerca Corrente 2023.

Keywords:
Missing evidence
Rehabilitation
Outcome non-reporting bias

Topics:
Research methodology, knowledge translation & implementation science

Did this work require ethics approval? No

Reason: The work is a cross-sectional meta-research study and thus does not require ethics approval because it is a secondary paper based on the results of other papers and not directly working on humans.