File
Moss P1, Barnett-Harris A1, Lee D1, Gupta K1, Sievers N1, Pritchard S1, Te M2, Blackstock F1,2
1Australian Physiotherapy Council, Richmond, Australia, 2Western Sydney University, Campbelltown, Australia
Background: The Australian Physiotherapy Council (the Council) assessment of overseas-qualified physiotherapists seeking registration in Australia requires candidates to demonstrate clinical competence in the core areas of practice: cardiorespiratory, neurological and musculoskeletal. To date, this has been completed through clinical assessments with patients in hospital settings. Sourcing and scheduling clinical assessments in collaborating healthcare institutions has become increasingly difficult, resulting in excessive wait times for candidates.
Simulation-based education has been shown to successfully replace 25% of 'real-life' clinical physiotherapy placement time, and simulation is now widely used across entry-level physiotherapy curricula in Australia. However, it is unknown whether a simulated experience enables valid assessment of the clinical competence of overseas-qualified physiotherapists seeking registration in Australia.
Purpose: To evaluate whether simulation-based clinical assessment is equivalent to the current hospital-based 'real-life' clinical assessment used by the Council. The study was carried out in two parts: an initial pilot study with 25 participants to evaluate proof-of-concept and feasibility, followed by a larger study comprising 150 participants.
Methods: Simulation-based assessment was carried out in two purpose-designed locations (Melbourne and Sydney). Western Sydney University Human Research Ethics Committee approval was obtained (H12123 and H12518). Candidates awaiting clinical assessments were invited to participate in queue order.
In Study 1, participants completed, in random order, three simulation-based assessments (one each core area) and three core area-matched 'real-life' clinical assessments.
In Study 2, candidates completed three simulation-based assessments (one each core area) and one 'real-life' assessment (in a randomly allocated core area). All simulation-based assessments were completed on the same day in randomly allocated order of completion. Timing of 'real-life' clinical assessments was dependent on healthcare facility availability. In simulation, purpose-written, competency mapped, peer-reviewed scenarios were portrayed by simulated patients. Simulation-based assessments were conducted in a simulated clinical setting with equipment and medical notes that replicated the real environment. Candidates were assessed using standard Council procedures for both assessment types. Pearson's Chi2 compared equivalence in pass/fail rates of the assessment types, and binary logistic regression examined predictors of performance in the 'real-life' clinical assessment.
Results: Interim results as of 07/09/18 are presented for 113 matched datasets (40 cardiorespiratory, 36 neurological and 37 musculoskeletal), full results to be presented at the conference.
Overall there was good equivalence. A similar number of passes and fails were observed: 48.7% and 45.2% pass for real-clinical and simulation-based assessments respectively. Overall 64.5% of candidates' pass/fail result matched in both simulation-based and real assessment (X2(3)=9.57, p=0.002). Participants who failed the 'real-life' clinical assessment scored significantly lower in the simulation-based assessment (Kruskal-Wallis: p 0.001). Pass/fail status in the simulation-based assessment was the best predictor of matched 'real-life' assessment outcome (odds-ratio 3.776, p 0.001). Location of assessment was also a significant predictor (odds-ratio 0.418, p=0.035), while core practice area was not (p=0.901).
Conclusion(s): Simulation-based clinical assessments have good equivalence to 'real-life' clinical assessments when evaluating overseas-qualified physiotherapists' competence to practice in Australia.
Implications: Carefully planned, authentic immersive simulation-based clinical assessments may improve the efficiency and standardisation of competency to practice assessment for overseas-qualified physiotherapists seeking Australian registration.
Keywords: Simulation, assessment, competency
Funding acknowledgements: This project was funded by the Australian Physiotherapy Council with in kind contribution from Western Sydney University
Simulation-based education has been shown to successfully replace 25% of 'real-life' clinical physiotherapy placement time, and simulation is now widely used across entry-level physiotherapy curricula in Australia. However, it is unknown whether a simulated experience enables valid assessment of the clinical competence of overseas-qualified physiotherapists seeking registration in Australia.
Purpose: To evaluate whether simulation-based clinical assessment is equivalent to the current hospital-based 'real-life' clinical assessment used by the Council. The study was carried out in two parts: an initial pilot study with 25 participants to evaluate proof-of-concept and feasibility, followed by a larger study comprising 150 participants.
Methods: Simulation-based assessment was carried out in two purpose-designed locations (Melbourne and Sydney). Western Sydney University Human Research Ethics Committee approval was obtained (H12123 and H12518). Candidates awaiting clinical assessments were invited to participate in queue order.
In Study 1, participants completed, in random order, three simulation-based assessments (one each core area) and three core area-matched 'real-life' clinical assessments.
In Study 2, candidates completed three simulation-based assessments (one each core area) and one 'real-life' assessment (in a randomly allocated core area). All simulation-based assessments were completed on the same day in randomly allocated order of completion. Timing of 'real-life' clinical assessments was dependent on healthcare facility availability. In simulation, purpose-written, competency mapped, peer-reviewed scenarios were portrayed by simulated patients. Simulation-based assessments were conducted in a simulated clinical setting with equipment and medical notes that replicated the real environment. Candidates were assessed using standard Council procedures for both assessment types. Pearson's Chi2 compared equivalence in pass/fail rates of the assessment types, and binary logistic regression examined predictors of performance in the 'real-life' clinical assessment.
Results: Interim results as of 07/09/18 are presented for 113 matched datasets (40 cardiorespiratory, 36 neurological and 37 musculoskeletal), full results to be presented at the conference.
Overall there was good equivalence. A similar number of passes and fails were observed: 48.7% and 45.2% pass for real-clinical and simulation-based assessments respectively. Overall 64.5% of candidates' pass/fail result matched in both simulation-based and real assessment (X2(3)=9.57, p=0.002). Participants who failed the 'real-life' clinical assessment scored significantly lower in the simulation-based assessment (Kruskal-Wallis: p 0.001). Pass/fail status in the simulation-based assessment was the best predictor of matched 'real-life' assessment outcome (odds-ratio 3.776, p 0.001). Location of assessment was also a significant predictor (odds-ratio 0.418, p=0.035), while core practice area was not (p=0.901).
Conclusion(s): Simulation-based clinical assessments have good equivalence to 'real-life' clinical assessments when evaluating overseas-qualified physiotherapists' competence to practice in Australia.
Implications: Carefully planned, authentic immersive simulation-based clinical assessments may improve the efficiency and standardisation of competency to practice assessment for overseas-qualified physiotherapists seeking Australian registration.
Keywords: Simulation, assessment, competency
Funding acknowledgements: This project was funded by the Australian Physiotherapy Council with in kind contribution from Western Sydney University
Topic: Education; Globalisation: health systems, policies & strategies; Research methodology & knowledge translation
Ethics approval required: Yes
Institution: Western Sydney University
Ethics committee: Western Sydney University Human Research Ethics Committee
Ethics number: H121213 and H12518
All authors, affiliations and abstracts have been published as submitted.
