File
O'Connor A1,2, Krucien N3, Cantillon P4, Parker M5, McCurtin A1,2
1University of Limerick, School of Allied Health, Limerick, Ireland, 2Health Research Institute, University of Limerick, Limerick, Ireland, 3Health Economics Research Unit, University of Aberdeen, Aberdeen, United Kingdom, 4National University of Ireland, Department of General Practice, Galway, Ireland, 5University of Limerick, Department of Physical Education and Sports Science, Limerick, Ireland
Background: Performance-based assessment (PBA) is a complex process that is integral to determining students' readiness for independent practice. Several challenges exist regarding this process and stakeholder input from clinical educators and physiotherapy students could provide valuable insight which may further its development. Evidence of stakeholder opinion is limited. While some qualitative studies have identified challenges, prioritisation of these needs is critical, particularly in the clinical learning environment, where the struggle between service delivery and education provision is one of the main barriers to undertaking student supervision.
Purpose: The aim of this study was to determine stakeholders' preferences towards optimisation of the physiotherapy PBA process in the Republic of Ireland. Three stakeholder groups were involved; physiotherapy undergraduate students, clinical educators (physiotherapists whose primary role is service provision but who also undertake student supervision) and practice tutors (physiotherapists employed in dedicated educational roles to support students and clinical educators).
Methods: Discrete Choice Experiment (DCE) was the methodology used. DCE is a statistical method providing a means of determining choice behaviour of participants. Using this technique, participants engage in a prioritisation exercise, employing a system of “trade-offs” which reflect their preferences for various components of a process, known as attributes and attribute levels. Attributes in this study were determined from focus group interviews conducted with the three stakeholder groups (n=79) which identified challenges of the PBA process. Based on this, four attributes were included. These were: 1) Assessment tool 2) Grading method 3) Assessors 4) Feedback mechanisms. Each attribute was defined by four levels which described hypothetical solutions to develop that aspect of the process. These attributes and levels were translated to a web-based DCE survey (www.surveymonkey.com) which was disseminated nationally via gatekeepers at Higher Education Institutions and professional bodies to students and educators respectively. A multinomial logit model was used to analyse the data.
Results: Sixty-seven students, 47 practice tutors and 113 clinical educators completed the survey. The most desirable developments identified by students and clinical educators were increased clarity of grading criteria in the assessment tool and the involvement of two assessors in student assessment (p≤0.05). All three groups identified weekly verbal feedback to students as a further significant priority (p≤0.05). Practice tutors also prioritised reducing repetition in the assessment tool (p≤0.05).
Conclusion(s): Similarities in the preferences expressed by stakeholder groups provide a common foundation for accelerating positive change in PBA in the Republic of Ireland (ROI). These centre on refinement of the assessment tool, core assessors and verbal feedback mechanisms. DCE was highlighted as a robust methodology, worthy of consideration by education providers seeking to address similar research questions.
Implications: A review of local practice education guidelines and policy is necessary in light of our findings. Currently there are no mandatory requirements regarding core assessors in practice education, highlighting the need for review. This study is important because it provides stakeholder-informed, priority-based guidance towards the development of PBA for physiotherapy education providers. While this study was conducted in the ROI, it has global implications given the international similarities of the PBA process in physiotherapy education.
Keywords: Performance-based assessment, practice education, physiotherapy students
Funding acknowledgements: This work was supported by Health Research Board, Ireland [HPF-2016- 1695].
Purpose: The aim of this study was to determine stakeholders' preferences towards optimisation of the physiotherapy PBA process in the Republic of Ireland. Three stakeholder groups were involved; physiotherapy undergraduate students, clinical educators (physiotherapists whose primary role is service provision but who also undertake student supervision) and practice tutors (physiotherapists employed in dedicated educational roles to support students and clinical educators).
Methods: Discrete Choice Experiment (DCE) was the methodology used. DCE is a statistical method providing a means of determining choice behaviour of participants. Using this technique, participants engage in a prioritisation exercise, employing a system of “trade-offs” which reflect their preferences for various components of a process, known as attributes and attribute levels. Attributes in this study were determined from focus group interviews conducted with the three stakeholder groups (n=79) which identified challenges of the PBA process. Based on this, four attributes were included. These were: 1) Assessment tool 2) Grading method 3) Assessors 4) Feedback mechanisms. Each attribute was defined by four levels which described hypothetical solutions to develop that aspect of the process. These attributes and levels were translated to a web-based DCE survey (www.surveymonkey.com) which was disseminated nationally via gatekeepers at Higher Education Institutions and professional bodies to students and educators respectively. A multinomial logit model was used to analyse the data.
Results: Sixty-seven students, 47 practice tutors and 113 clinical educators completed the survey. The most desirable developments identified by students and clinical educators were increased clarity of grading criteria in the assessment tool and the involvement of two assessors in student assessment (p≤0.05). All three groups identified weekly verbal feedback to students as a further significant priority (p≤0.05). Practice tutors also prioritised reducing repetition in the assessment tool (p≤0.05).
Conclusion(s): Similarities in the preferences expressed by stakeholder groups provide a common foundation for accelerating positive change in PBA in the Republic of Ireland (ROI). These centre on refinement of the assessment tool, core assessors and verbal feedback mechanisms. DCE was highlighted as a robust methodology, worthy of consideration by education providers seeking to address similar research questions.
Implications: A review of local practice education guidelines and policy is necessary in light of our findings. Currently there are no mandatory requirements regarding core assessors in practice education, highlighting the need for review. This study is important because it provides stakeholder-informed, priority-based guidance towards the development of PBA for physiotherapy education providers. While this study was conducted in the ROI, it has global implications given the international similarities of the PBA process in physiotherapy education.
Keywords: Performance-based assessment, practice education, physiotherapy students
Funding acknowledgements: This work was supported by Health Research Board, Ireland [HPF-2016- 1695].
Topic: Education; Education: methods of teaching & learning; Education
Ethics approval required: Yes
Institution: University of Limerick
Ethics committee: Education and Health Science Research Ethics Committtee (EHSREC)
Ethics number: 2016_02_03_EHS
All authors, affiliations and abstracts have been published as submitted.
