File
Goodwin V1, Hill J1, Fullam J1, Finning K1, Pentecost C1, Richards D1
1University of Exeter, Exeter, United Kingdom
Background: Rehabilitation is a complex process and randomised controlled trials of rehabilitation interventions are increasing in number. The Medical Research Council Framework for Developing and Evaluating Complex Interventions was developed to optimise the likelihood that new interventions are not rejected as being ineffective when inadequate effort has been made in the development of the intervention. However, there has been no evaluation of the impact of using the development component of the framework on trial outcomes.
Purpose: This study aimed to establish treatment success rates in rehabilitation trials funded by the NIHR Health Technology Assessment (HTA) programme, the leading funder of applied health research in the UK, and examine any relationship between treatment success and the quality of intervention development work undertaken.
Methods: The NIHR HTA portfolio was searched for all completed definitive randomised controlled trials of physiotherapy, occupational therapy or speech and language therapy from inception to July 2016. Treatment success, as defined by Djulbegovic et al, was categorised for the primary outcome as:
1) statistically significant in favour of the new treatment,
2) statistically significant in favour of the control treatment
3) true negative (no difference),
4) truly inconclusive,
5) inconclusive in favour of new treatment or
6) inconclusive in favour of the control treatment.
Detailed textual data regarding any intervention development work were extracted from the trial reports and supporting publications and informed the development of a quality rating. Mixed methods integrative analysis was undertaken to explore the relationship between the quantitative and qualitative data using joint displays.
Results: Fifteen trials were included in the review with a total amount of over £11 million in research funding awarded. Of these, five trials reported a definitive finding, four of which were in favour of the 'new' intervention. Eight trials reported a true negative (no difference) outcome. A data-driven approach enabled the development of a quality rating of intervention development work. Integrative analysis indicated those with higher quality intervention development work were more likely to report treatment success, although older and possibly less well reported trials also reported effective interventions. Studies with low quality intervention development work were more likely to report a null result.
Conclusion(s): Despite much effort and funding, most NIHR-funded trials of rehabilitation report equivocal findings. Greater focus on high quality intervention development may reduce the likelihood of a null result in the definitive trial.
Implications: As this was an exploratory study, further work should be undertaken to establish the the validity of quality assessment of intervention development ratings. This said, researchers and funding agencies should not undervalue the potential benefit of high quality intervention development work prior to definitive randomised controlled trials to reduce the likelihood of a null outcome and improve current rates of treatment success.
Keywords: Mixed methods, Randomised controlled trials, Intervention development
Funding acknowledgements: This work was supported by the NIHR CLAHRC South West Peninsula.
Purpose: This study aimed to establish treatment success rates in rehabilitation trials funded by the NIHR Health Technology Assessment (HTA) programme, the leading funder of applied health research in the UK, and examine any relationship between treatment success and the quality of intervention development work undertaken.
Methods: The NIHR HTA portfolio was searched for all completed definitive randomised controlled trials of physiotherapy, occupational therapy or speech and language therapy from inception to July 2016. Treatment success, as defined by Djulbegovic et al, was categorised for the primary outcome as:
1) statistically significant in favour of the new treatment,
2) statistically significant in favour of the control treatment
3) true negative (no difference),
4) truly inconclusive,
5) inconclusive in favour of new treatment or
6) inconclusive in favour of the control treatment.
Detailed textual data regarding any intervention development work were extracted from the trial reports and supporting publications and informed the development of a quality rating. Mixed methods integrative analysis was undertaken to explore the relationship between the quantitative and qualitative data using joint displays.
Results: Fifteen trials were included in the review with a total amount of over £11 million in research funding awarded. Of these, five trials reported a definitive finding, four of which were in favour of the 'new' intervention. Eight trials reported a true negative (no difference) outcome. A data-driven approach enabled the development of a quality rating of intervention development work. Integrative analysis indicated those with higher quality intervention development work were more likely to report treatment success, although older and possibly less well reported trials also reported effective interventions. Studies with low quality intervention development work were more likely to report a null result.
Conclusion(s): Despite much effort and funding, most NIHR-funded trials of rehabilitation report equivocal findings. Greater focus on high quality intervention development may reduce the likelihood of a null result in the definitive trial.
Implications: As this was an exploratory study, further work should be undertaken to establish the the validity of quality assessment of intervention development ratings. This said, researchers and funding agencies should not undervalue the potential benefit of high quality intervention development work prior to definitive randomised controlled trials to reduce the likelihood of a null outcome and improve current rates of treatment success.
Keywords: Mixed methods, Randomised controlled trials, Intervention development
Funding acknowledgements: This work was supported by the NIHR CLAHRC South West Peninsula.
Topic: Disability & rehabilitation
Ethics approval required: No
Institution: NA
Ethics committee: NA
Reason not required: This study was secondary analysis of published studies
All authors, affiliations and abstracts have been published as submitted.
