Castellini G1,2, Gianola S1, Corbetta D3, Moja L1,2
1IRCCS Istituto Ortopedico Galezzi, Unità di Epidemiologia Clinica, Milano, Italy, 2Università Degli Studi di Milano, Department of Biomedical Sciences for Health, Milano, Italy, 3Università Vita Salute San Raffaele, Milano, Italy
Background: An observed statistically significant difference between two interventions does not necessarily imply that this difference is clinically important for patients.
Purpose: We aimed to interpret the results of published trials in terms of clinical relevance and statistical significance. In particular, we assessed if randomized controlled trials (RCTs) achieved the minimal important difference (MID) planned in the sample size calculation.
Methods: We performed a cross-sectional study of all RCTs included in Cochrane Systematic Reviews (SRs) published until April 2017, focusing on the efficacy of rehabilitation interventions for low back pain (LBP). RCTs having sample size calculation and a planned MID were considered. Using an ad hoc checklist, two independent investigators collected the study characteristics, the mean differences and their confidence intervals for primary continuous outcomes and the planned MID. The primary outcome was to identify the number of RCTs classified as statistically significant and clinically relevant, statistically significant but not clinically relevant, clinically relevant but not statistically significant, and both not statistically and clinically significant. The secondary outcome was the same categorization for all compared interventions.
Results: From 20 eligible SRs including 101 RCTs, we identified 42 RCTs (42%) involving 4966 patients and 81 intervention comparisons. Overall, we found that more than one-third of RCTs (36%, n=15) were both statistically and clinically significant whereas few (24%, n=10) were statistically significant but not clinically relevant. Taking into account all comparisons of multiple arm trials the scenarios were similar. However, an half of the positive comparisons compared an active treatment (e.g., manipulation) against an inert treatment (e.g., sham manipulation) while the remaining comparisons explored active head-to-head comparisons.
Conclusion(s): Despite the amount of efforts in conducting RCTs in LBP rehabilitation, just few of them reached both statistically significant and clinically relevant results and the majority of these were compared against open-and-shut interventions.
Implications: It is essential to consider the clinical relevance in order to involve patient perspective and transfer research results into clinical practice.
Keywords: Clinical relevance, statistical significance, randomized controlled trials
Funding acknowledgements: The work was supported by the Italian Ministry of Health (L3026) and by Italian Ministry of Health (Giovani Ricercatori GR-2011-02348048).
Purpose: We aimed to interpret the results of published trials in terms of clinical relevance and statistical significance. In particular, we assessed if randomized controlled trials (RCTs) achieved the minimal important difference (MID) planned in the sample size calculation.
Methods: We performed a cross-sectional study of all RCTs included in Cochrane Systematic Reviews (SRs) published until April 2017, focusing on the efficacy of rehabilitation interventions for low back pain (LBP). RCTs having sample size calculation and a planned MID were considered. Using an ad hoc checklist, two independent investigators collected the study characteristics, the mean differences and their confidence intervals for primary continuous outcomes and the planned MID. The primary outcome was to identify the number of RCTs classified as statistically significant and clinically relevant, statistically significant but not clinically relevant, clinically relevant but not statistically significant, and both not statistically and clinically significant. The secondary outcome was the same categorization for all compared interventions.
Results: From 20 eligible SRs including 101 RCTs, we identified 42 RCTs (42%) involving 4966 patients and 81 intervention comparisons. Overall, we found that more than one-third of RCTs (36%, n=15) were both statistically and clinically significant whereas few (24%, n=10) were statistically significant but not clinically relevant. Taking into account all comparisons of multiple arm trials the scenarios were similar. However, an half of the positive comparisons compared an active treatment (e.g., manipulation) against an inert treatment (e.g., sham manipulation) while the remaining comparisons explored active head-to-head comparisons.
Conclusion(s): Despite the amount of efforts in conducting RCTs in LBP rehabilitation, just few of them reached both statistically significant and clinically relevant results and the majority of these were compared against open-and-shut interventions.
Implications: It is essential to consider the clinical relevance in order to involve patient perspective and transfer research results into clinical practice.
Keywords: Clinical relevance, statistical significance, randomized controlled trials
Funding acknowledgements: The work was supported by the Italian Ministry of Health (L3026) and by Italian Ministry of Health (Giovani Ricercatori GR-2011-02348048).
Topic: Musculoskeletal: spine; Musculoskeletal: spine; Disability & rehabilitation
Ethics approval required: No
Institution: none
Ethics committee: none
Reason not required: methodological research (review) that not recruited participants and not performed prospective interventions
All authors, affiliations and abstracts have been published as submitted.
