Andrews K.1, Corrigan R.1, Paton I.1, O'Shea S.1
1Charles Sturt University, School of Community Health, Albury, Australia
Background: Practical examinations are commonly used in Physiotherapy education to assess students' clinical reasoning and technical skills related to clinical scenarios. The challenge for examiners is to provide consistent, fair and accurate assessment, usually according to predetermined criteria. At Charles Sturt University, multi-campus delivery of the Physiotherapy program was identified as a factor challenging the reliability of practical examination marking. Therefore, re-thinking established moderation practices for practical examinations was required.
Purpose: To trial and evaluate a real-time cross-campus moderation approach to practical examination in the Physiotherapy program at Charles Sturt University.
Methods: Prior to examinations, all assessors reviewed the practical exam procedures, marking rubric and subject learning objectives. Real-time moderation was undertaken via videoconference (Skype). The moderation process was undertaken on the first day of the practical examination period, and included 10% of the total student cohort. The assessor on each campus conducted three (3) practical examinations with the assessor on the other campus observing and marking in real-time via Skype. At the end of each of the moderated practical examinations, the examiner and the observer would compare and discuss their marks for each component of the rubric, with final marks determined via consensus. The observer also provided peer feedback regarding conduct of the examination. The process was iterative, whereby examiners would carry forward the discussion and apply feedback to subsequent examinations. The initial trial was evaluated through student and examiner feedback, and comparison of the marks from each campus.
Results: Students initially reported feeling concerned about being double marked. However, they reported that the actual process was not obtrusive. Examiners relished the opportunity for real-time feedback and peer review of their performance; but reported the process provided additional challenges relating to managing multiple tasks concurrently.
A small difference between campuses was found in the spread of marks, which was consistent with performance on other assessment items within the subject, as well as in other subjects. The real time moderation process added an extra five (5) minutes per student to the examination process, and the videoconference platform whilst simple and cost-efficient, had its limitations (poor sound & picture quality, variable internet connections, availability of suitable devices). The process also led to modification of the marking rubric for future examinations, and enabled identification of areas within the subjects design and delivery that may benefit from further development.
Conclusion(s): Real-time moderation was a relatively simple approach to practical examination quality assurance, and provided additional benefits in relation to assessor peer review. However, further evaluation is required to determine how this process influences inter-examiner reliability across campuses.
Implications: Clear and robust assessment and moderation processes are required to ensure fair and equitable student assessment, as well as ongoing staff development and subject design.
Funding acknowledgements: Nil
Topic: Education
Ethics approval: Ethics approval was not required.
All authors, affiliations and abstracts have been published as submitted.
