A. Barnett-Harris1, D. Lee1, L. Percy1, R. Reichman1
1Australian Physiotherapy Council, Melbourne, Australia
Background: The COVID-19 pandemic disrupted the activities of many accreditation authorities worldwide. The Australian Physiotherapy Council (the Council) was severely affected, with accreditation of programs stalled and assessments of overseas qualified physiotherapists suspended. Whilst the duration of impact was unknown, the Council acted swiftly in commencing a project to rapidly develop process changes that ensure operations could continue, whilst still ensuring quality and robustness of its activities were not compromised.
Purpose: The project aimed to build more resilient, adaptable, capable, and future-ready processes to ensure the continuity of services, regardless of environment or context.
Methods: Development of new processes was done through multiple stages: Stage 1. Environmental review performed to seek relevant approaches and innovations. Stage 2. Existing end to end business processes reviewed to identify areas disrupted by COVID-19. Stage 3. Analysis of process changes in past 3 years to examine where recent innovations can be expanded/enhanced. Stage 4. Rapid prototyping involving consultation with key stakeholders whilst ensuring integrity of processes. For higher stakes processes: Stage 5. Pilot changes to sample customers. Stage 6. Qualitative and quantitative measurement of pilot through questionnaires and analysis of data reports.
Results: Previous years’ investment in IT and development of its unique “Village” organisational culture facilitated a rapid and successful shift to fully remote operations, in line with government requirements. The “Village” value of nimble and brave fostered a culture of adaptability and innovativeness that enabled the rapid prototyping and testing of process changes.
The Council pivoted to online accreditation operations, leveraging off recent investment in an online application to manage accreditation processes, and implementing virtual site visits. There was a high acceptance rate: 9 of 10 university education programs.
The Council also pivoted to remotely invigilated MCQ assessment, developed in collaboration with its exam partner and piloted in July 2020. There was a 50% uptake (79 of 158) of candidates initially scheduled to a physical assessment venue, which was an appropriate sample size for the pilot. Data analysis of results using Rasch analysis indicate no variation in item and cohort performance as compared to previous assessments in physical venues, and no reported incidences of cheating. Completed questionnaires of the sample (n=51) indicated that candidates were mostly satisfied with the remote assessment, with 90% agreeing that more remotely invigilated sessions should be offered in the future.
The Council pivoted to online accreditation operations, leveraging off recent investment in an online application to manage accreditation processes, and implementing virtual site visits. There was a high acceptance rate: 9 of 10 university education programs.
The Council also pivoted to remotely invigilated MCQ assessment, developed in collaboration with its exam partner and piloted in July 2020. There was a 50% uptake (79 of 158) of candidates initially scheduled to a physical assessment venue, which was an appropriate sample size for the pilot. Data analysis of results using Rasch analysis indicate no variation in item and cohort performance as compared to previous assessments in physical venues, and no reported incidences of cheating. Completed questionnaires of the sample (n=51) indicated that candidates were mostly satisfied with the remote assessment, with 90% agreeing that more remotely invigilated sessions should be offered in the future.
Conclusion(s): An adaptable, flexible, innovative culture enabled rapid development of significant process changes. Investment in IT laid the foundation for future-ready processes. Strong stakeholder relationships were vital in ensuring collaboration and engagement in the midst of a pandemic. The process changes enabled the Council to minimise disruption caused by COVID-19.
Implications: Rapid changes are not typical in the regulatory space. The Council’s approach to these changes whilst balancing risk and ensuring quality and robustness, may be used as an example by other authorities, to develop their own approaches whilst navigating operational disruption in challenging circumstances.
Funding, acknowledgements: The Council was the primary funder, to innovate and find sustainable approaches to Accreditation and Assessment in a COVID-19 world.
Keywords: COVID-19, rapid prototyping, innovation
Topic: COVID-19
Did this work require ethics approval? No
Institution: N/A
Committee: N/A
Reason: There were internal risk management mechanisms and governance oversight to innovate our established methods of accreditation and assessment.
All authors, affiliations and abstracts have been published as submitted.
