Development of a quality assessment framework for digital health technology in physiotherapy: An E-Delphi study

Kasper Recourt, Cindy Veenhof, Sander van de Hoef, Corelien Kloek
Purpose:

To develop an easy, quick-to-score, quality assessment framework for digital health technologies in physiotherapy incorporating the perspectives of experts.

Methods:

A national e-Delphi procedure was conducted. Experts in physical therapy, research, health technology development, education (physical therapy) and policy were included. The Delphi procedure consisted of four rounds and each round consisted of a questionnaire and an anonymized report. A questionnaire was developed based on existing quality assessment frameworks as the CEN-ISO/TS 82304-2, MARS tool and frameworks from private parties. All items that could be scored without specific information that was only available via the developer, were included in the survey. The first three rounds consisted of questions regarding: Proposed criteria for the quality assessment framework and presentation of the results. The fourth round a conceptual rubric was proposed. Consensus was reached when >70% of the experts agreed or a 7/10 or higher was scored on a criterium.

Results:

Thirty experts with backgrounds in the physiotherapy, education, technology and policy domain, participated. This Delphi study resulted in a quality assessment framework, a presentation label and a rubric. (1) The quality assessment framework contains 15 criteria that assess the technology in three categories: Health & Safety, Usability and Data safety. Regarding visualization, the experts found it important to present an overall score, sub-scores per category and user-reviews. They  also agreed on the proposed rubric which resulted in a five-star scoring system. 

Conclusion(s):

This study showed which criteria experts found important to determine the quality of the technology. This is the first step in developing a quick and easy-to-use framework for the assessment of digital health technology. Future research should indicate whether this quality assessment framework is usable and determine validity, sensitivity, specificity and responsiveness.

Implications:

This Delphi study showed which criteria, regarding healthcare technology, experts found important. This framework could be used by independent parties to assess the technologies and the resulting scores could be used on existing or new platforms with overviews of digital health technologies to inform physiotherapists. Nevertheless, it remains important to consider whether the assessed technology suits the digital – and  health literacy of the patient and healthcare professional and whether the technology supports them in reaching their treatment goals. A major challenge is to keep scores up-to-date, since technologies are rapidly evolving.

Funding acknowledgements:
No funding was received for this project.
Keywords:
Digital health
Quality assessment
Delphi study
Primary topic:
Innovative technology: information management, big data and artificial intelligence
Second topic:
Other
Did this work require ethics approval?:
No
Name the institution and ethics committee that approved your work:
The Ethical Committee Research Healthcare Domain of the HU University of Applied Sciences Utrecht stated ethics approval was not required
Provide the ethics approval number:
File number 176-000-2022
Has any of this material been/due to be published or presented at another national or international conference prior to the World Physiotherapy Congress 2025?:
Yes

Back to the listing