File
Pellicciari L1, Piscitelli D2,3, Caselli S4,5, La Porta F6
1IRCCS San Raffaele Pisana, Department of Neurorehabilitation, Rome, Italy, 2McGill University, School of Physical and Occupational Therapy, Montreal, Canada, 3University of Milano-Bicocca, School of Medicine and Surgery, Milan, Italy, 4Azienda Ospedaliero-Universitaria di Modena, Dipartimento di Neuroscienze, Modena, Italy, 5Università degli Studi di Milano-Bicocca, Scuola di Dottorato in Sanità Pubblica, Milan, Italy, 6Dipartimento di Emergenza, Unità Operativa Complessa di Medicina Riabilitativa e Neuroriabilitazione, Bologna, Italy
Background: Using reliable and valid outcome measurements to assess the risk of falling is crucial both in research and clinical practice. In several Italian hospitals, the Conley Scale (CS) is used for this purpose. It comprises six dichotomous items, with a total score ranging from 0 to 10. A total score below two indicates lower risk of falling. The psychometric properties of this tool were assessed in patients admitted to a general hospital using both Classical Theory Test (CTT) and Rasch Measurement Theory (RMT). However, as the tool is employed also in rehabilitation, it is mandatory to investigate its psychometric profile also in this setting.
Purpose: The aim of this study was to assess the psychometric properties of the CS in patients admitted to a rehabilitation ward through CTT and RMT.
Methods: The CS was administered to 946 (514 [54.3%] female; mean age ± SD: 71.9±13.92) patients admitted to a rehabilitation ward of the “San Giuseppe” hospital in Empoli, Italy, across two years. Confirmatory Factor Analysis (CFA) and Rasch analysis (RA) were undertaken to assess its dimensionality, internal construct validity, reliability indexes, differential item functioning (DIF), and local dependence.
Results: CFA showed its dimensionality after adjusting local dependency between item#5 and item#6 (CFI=0.977>0.95; TLI=0.957>0.95, RMSEA=0.035 0.05). However, the scale on entire sample did not fit the Rasch model (χ2=48.42; p=0.000003), even after minimizing for type-1 error, creating 4 mutually-exclusive random sub-samples of equal size (N=237), with several post-hoc modifications. All the base analyses showed misfit of the CS to the Rasch model and reliability values well below the minimum cutoff for group measurement ( 0.70). In three out of four samples, there was local dependency between item#5 and item#6 (thus confirming the CFA findings). There was also evidence of DIF. After extensive modifications of the scale within each sample aiming at resolve local dependency and DIF, local dependency was resolved, but DIF resulted not solvable in two of the three subsamples. In despite these modifications, fitness to the Rasch model was rejected for all the stepwise solutions adopted and the reliability levels remained well below the minimum recommended cut-off for group measurement ( 0.70).
Conclusion(s): Although the CS is unidimensional, it does not meet the minimum criteria for meaningful measurement according to the RMT. Thus, its total score is misleading, since the distances between each item have no real meaning. These results match those obtained also in a general hospital samples. Hence, even in the rehabilitation setting using the CS to identify patient at risk of falling is not recommended.
Implications: Administering the CS in order to identify patients at risk of falling may result in a high likelihood of identifying as "at risk of falling" patients who are not (causing a useless consumption of resources), or, above all, a high probability of causing the clinicians to fail, leading them to consider as "not at risk" patients who actually are and exposing them to the risk of falling. Is it not the time to throw in the towel for the CS also in the rehabilitation setting?
Keywords: Accidental falls, Psychometrics, Item Response Theory.
Funding acknowledgements: No funding to declare.
Purpose: The aim of this study was to assess the psychometric properties of the CS in patients admitted to a rehabilitation ward through CTT and RMT.
Methods: The CS was administered to 946 (514 [54.3%] female; mean age ± SD: 71.9±13.92) patients admitted to a rehabilitation ward of the “San Giuseppe” hospital in Empoli, Italy, across two years. Confirmatory Factor Analysis (CFA) and Rasch analysis (RA) were undertaken to assess its dimensionality, internal construct validity, reliability indexes, differential item functioning (DIF), and local dependence.
Results: CFA showed its dimensionality after adjusting local dependency between item#5 and item#6 (CFI=0.977>0.95; TLI=0.957>0.95, RMSEA=0.035 0.05). However, the scale on entire sample did not fit the Rasch model (χ2=48.42; p=0.000003), even after minimizing for type-1 error, creating 4 mutually-exclusive random sub-samples of equal size (N=237), with several post-hoc modifications. All the base analyses showed misfit of the CS to the Rasch model and reliability values well below the minimum cutoff for group measurement ( 0.70). In three out of four samples, there was local dependency between item#5 and item#6 (thus confirming the CFA findings). There was also evidence of DIF. After extensive modifications of the scale within each sample aiming at resolve local dependency and DIF, local dependency was resolved, but DIF resulted not solvable in two of the three subsamples. In despite these modifications, fitness to the Rasch model was rejected for all the stepwise solutions adopted and the reliability levels remained well below the minimum recommended cut-off for group measurement ( 0.70).
Conclusion(s): Although the CS is unidimensional, it does not meet the minimum criteria for meaningful measurement according to the RMT. Thus, its total score is misleading, since the distances between each item have no real meaning. These results match those obtained also in a general hospital samples. Hence, even in the rehabilitation setting using the CS to identify patient at risk of falling is not recommended.
Implications: Administering the CS in order to identify patients at risk of falling may result in a high likelihood of identifying as "at risk of falling" patients who are not (causing a useless consumption of resources), or, above all, a high probability of causing the clinicians to fail, leading them to consider as "not at risk" patients who actually are and exposing them to the risk of falling. Is it not the time to throw in the towel for the CS also in the rehabilitation setting?
Keywords: Accidental falls, Psychometrics, Item Response Theory.
Funding acknowledgements: No funding to declare.
Topic: Outcome measurement; Older people
Ethics approval required: No
Institution: Azienda USL Toscana Centro
Ethics committee: N/A
Reason not required: This project based on a tailored clinical daily practice; an informed consent was obtained from participants to use their data.
All authors, affiliations and abstracts have been published as submitted.
