M. Unger1, L. Manas1, L. Keiller2
1Stellenbosch University, Rehabilitation and Health Sciences, Cape Town, South Africa, 2Stellenbosch University, Learning Technology Systems, Cape Town, South Africa
Background: Undergraduate physiotherapy students have to learn to effectively perform in excess of 200 evaluation and treatment techniques needed for successful practice. Faculty often question the validity of the current Objective Structured Practical Exam (OSPE) format of assessment and students are struggling to meet the minimum standard before entering and applying these techniques in the work place. An alternative strategy, a self-recorded video portfolio was implemented to get students to practice these techniques more consistently and to help them reach satisfactory levels of proficiency.
Purpose: To investigate the effect, and student perceptions, of a peer-assessed video e-portfolio intervention on learning and assessment of physiotherapy foundational practical techniques. The secondary objective was to determine the reliability of peer assessment..
Methods: A mixed methods approach with purposive sampling was used. All second year students (n=67) after submitting 23 techniques to their e-portfolios were invited to complete a Likert-scale rated and open-ended self-perception questionnaire. Retrospective analysis using Pearson’s correlation coefficient compared student peer-assessed e-portfolio and final OSPE scores. Repeated measures ANOVA compared performance across 4 OSPE assessments. Intra-class Correlation Coefficients (ICC agreement) using a two-way mixed effects model, was used to explore reliability of peer assessment.
Results: The response rate was fair (62.7%) with age (19-26 years) and gender (5 males and 37 females) demographics representative of the class. Student responses suggested the intervention made them practice more (87.2%), helped them do better in the OSPE (77.5%) and made them reflect on their own performance (71.8%). Overall OSPE scores improved significantly throughout the year (p=0.021). Contrary to our hypothesis, the final marks for the e-portfolio was almost identical to the final OSPE scores (p=0.96).
Sixty five percent of students rated peer-assessment as consistently fair. Reliability of peer-assessment compared to faculty was poor with an ICC of .616, but this improved significantly (ICC=.938) when a benchmark video was provided prior to peer-assessment.
Sixty five percent of students rated peer-assessment as consistently fair. Reliability of peer-assessment compared to faculty was poor with an ICC of .616, but this improved significantly (ICC=.938) when a benchmark video was provided prior to peer-assessment.
Conclusion(s): A video portfolio in which students weekly practiced, self-recorded and peer-assessed selected physiotherapy foundational practical techniques resulted in students practicing more consistently. Students valued the intervention and it had a significant positive effect on their final OSPE scores, this despite more than 90% of the OSPE techniques differing from the techniques recorded for their e-portfolios.
Peer-assessment is reliable, with the proviso that a benchmark video is provided prior to assessment. The results from this study suggests a video e-portfolio may be a feasible alternative to the current stressful, time-consuming and expensive OSPE format however further investigation into its validity and predictive value of performance in the real world is recommended.
Peer-assessment is reliable, with the proviso that a benchmark video is provided prior to assessment. The results from this study suggests a video e-portfolio may be a feasible alternative to the current stressful, time-consuming and expensive OSPE format however further investigation into its validity and predictive value of performance in the real world is recommended.
Implications: Results from the current study suggest:
- self-recorded videos of foundational practical techniques that are also peer-assessed is an appropriate intervention to improve student engagement in their mastery of physiotherapy evaluation and treatment techniques
- further investigation into the predictive validity of peer-assessed e-portfolios of practical techniques is recommended to enable exploration of this as an alternative assessment strategy to the stressful, expensive and time consuming OSPE format, currently considered the gold-standard for practical skills assessment.
Funding, acknowledgements: This study was funded by the Stellenbosch University's Fund for Innovation and Research into Learning and Teaching (FIRLT)
Keywords: practical techniques, video e-portfolio, assessment
Topic: Education: methods of teaching & learning
Did this work require ethics approval? Yes
Institution: Stellenbosch University
Committee: Health Research Ethics Committee (HREC 1)
Ethics number: N18/09/093
All authors, affiliations and abstracts have been published as submitted.
