OSCE Feedback: A Randomized Trial of Effectiveness,Cost-Effectiveness and Student Satisfaction

Celia A. Taylor, Kathryn E. Green

School of Clinical and Experimental Medicine, University of Birmingham, Birmingham, UK.

DOI: 10.4236/ce.2013.46A002                

Abstract

Purpose: To develop two new types of clinical feedback for final year medical students using OSCE mark sheets and to evaluate their effectiveness, cost-effectiveness and student satisfaction in a randomized trial. Methods: A randomized trial was conducted with two groups (Cohort A and B) of students (n = 350) at the University of Birmingham (UK) participating in a two stage Objective Structured Clinical Examination (OSCE) (November 2011 and April 2012). Students were randomly assigned to receive one of three feedback interventions (skills-based, station-based, or both) after the November OSCE. Multivariate regression analysis was used to test if feedback intervention was a significant predictor of April OSCE score, while controlling for November OSCE score. Secondary outcomes were cost-effectiveness and student satisfaction. Results: Feedback group was not a significant predictor of April scores for Cohort B. In Cohort A, the station-based group did better than the group who received both types of feedback (2.8%, 95% CI 0.4% to 5.2%, p = 0.022). There was no difference between the skills-based and station-based groups. The cost of providing the station-based feedback was double of that for the skills-based. Questionnaires were received from 245 students (70%). Students who received both types of feedback were the most satisfied, followed by those in the station-based group. Conclusion: There was no consistent difference in effectiveness across the three trial groups. Students tended to prefer station-based feedback over skills-based feedback, but students found elements of the standard feedback more helpful than the feedback evaluated in this trial.

Keywords

Assessment; Medical Students; Objective Structured Clinical Examination; Feedback; Randomized Trial

Conflicts of Interest

The authors declare no conflicts of interest.

References

[1]	Bing-You, R. G., Paterson, J., & Levine, M. A. (1997). Feedback falling on deaf ears: Residents’ receptivity to feedback tempered by sender credibility. Medical Teacher, 19, 40-44. doi:10.3109/01421599709019346
[2]	Black, N. M. I., & Harden, R. M. (1986). Providing feedback to students on clinical skills by using the objective structured clinical examination. Medical Education, 20, 48-52. doi:10.1111/j.1365-2923.1986.tb01041.x
[3]	Boehler, M. L., Rogers, D. A., Schwind, C. J., Mayforth, R., Quin, J., Williams, R. G., et al. (2006). An investigation of medical student reactions to feedback: A randomised controlled trial. Medical Education, 40, 746-749. doi:10.1111/j.1365-2929.2006.02503.x
[4]	Boet, S., Sharma, S., Goldman, J., & Reeves, S. (2012). Review article: Medical education research: An overview of methods. Canadian Journal of Anesthesia, 59, 159-170. doi:10.1007/s12630-011-9635-y
[5]	Brown, N., & Cooke, L. (2009). Giving effective feedback to psychiatric trainees. Advances in Psychiatric Treatment, 15, 123-128. doi:10.1192/apt.bp.106.003293
[6]	Brukner, H., Altkorn, D. L., Cook, S., Quinn, M. T., & Mcnabb, W. L. (1999). Giving effective feedback to medical students: A workshop for faculty and house staff. Medical Teacher, 21, 161-165. doi:10.1080/01421599979798
[7]	Chowdhury, R. R., & Kalu, G. (2004). Learning to give feedback in medical education. The Obstetrician & Gynaecologist, 6, 243-247. doi:10.1576/toag.6.4.243.27023
[8]	Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Earlbaum.
[9]	De, S. K., Henke, P. K., Ailawadi, G., Dimick, J. B., & Colletti, L. M. (2004). Attending, house officer, and medical student perceptions about teaching in the third-year medical school general surgery clerkship. Journal of the American College of Surgeons, 199, 932-942. doi:10.1016/j.jamcollsurg.2004.08.025
[10]	Ende, J. (1983). Feedback in clinical medical education. Journal of the American Medical Association, 250, 777-781. doi:10.1001/jama.1983.03340060055026
[11]	Faul, F., Erdfelder, E., Buchner, A., & Lang, A. G. (2009). Statistical power analyses using G*Power 3.1: Tests for correlation and regression analyses. Behavior Research Methods, 41, 1149-1160. doi:10.3758/BRM.41.4.1149
[12]	General Medical Council (2009). Tomorrow’s doctors. London: General Medical Council.
[13]	Gigante, J., Dell, M., & Sharkey, A. (2011). Getting beyond “good job”: How to give effective feedback. Pediatrics, 127, 205-207. doi:10.1542/peds.2010-3351
[14]	Gil, D. H., Heins, M., & Jones, P. B. (1984). Perceptions of medical school faculty members and students on clinical clerkship feedback. Academic Medicine, 59, 856-864. doi:10.1097/00001888-198411000-00003
[15]	Hodder, R. V., Rivington, R. N., Calcutt, L. E., & Hart, I. R. (1989). The effectiveness of immediate feedback during the Objective Structured Clinical Examination. Medical Education, 23, 184-188. doi:10.1111/j.1365-2923.1989.tb00884.x
[16]	Hollingsworth, M., Richards, B. F., & Frye, A. W. (1994). Description of observer feedback in an objective structured clinical examination and effects on examinees. Teaching and Learning in Medicine, 6, 49-53. doi:10.1080/10401339409539643
[17]	Kluger, A., & DeNisi, A. (1996). The effects of feedback interventions on performance: A historical review, a meta-analysis, and a preliminary feedback intervention theory. Psychological Bulletin, 119, 254- 284. doi:10.1037/0033-2909.119.2.254
[18]	Livingston, S., & Zieky, M. (1982). Passing scores: A manual for setting standards of performance on educational and occupational tests. Princeton, NJ: Educational Testing Service.
[19]	Moher, D., Hopewell, S., Schulz, K. F., Montori, V., Gotzsche, P. C., Devereaux, P. J., et al. (2010). CONSORT 2010 Explanation and Elaboration: Updated guidelines for reporting parallel group randomised trials. British Medical Journal, 340, c.869. doi:10.1136/bmj.c869
[20]	Sargeant, J. M., Mann, K. V., Vander Vleuten, C. P., & Metsemakers, J. F. (2009). Reflection: A link between receiving and using assessment feedback. Advances in Health Science Education, 14, 399-410. doi:10.1007/s10459-008-9124-4
[21]	Schulz, K. F., Altman, D. G., & Moher, D. for the CONSORT Group (2010). CONSORT 2010 statement: Updated guidelines for reporting parallel group randomised trials. PLoS Medicine, 7, e1000251. doi:10.1371/journal.pmed.1000251
[22]	University of Washington (2004). Second year OSCEs: Preparing for the second year OSCEs. URL (last checked 14 September 2012). http://www.uwmedicine.org/Education/MD-Program/Current-Students/Curriculum/OSCE/Pages/Second-Year-OSCEs.aspx
[23]	Van De Ridder, J. M. M., Stokking, K. M., McGaghie, W. C., & Ten Cate, O. T. (2008). What is feedback in clinical education? Medical Education, 42, 189-197. doi:10.1111/j.1365-2923.2007.02973.x
[24]	Veloski, J., Boex, J. R., Grasberger, M. J., Evans, A., & Wolfson, D. B. (2006). Systematic review of the literature on assessment, feedback and physicians’ clinical performance: BEME Guide No. 7. Medical Teacher, 28, 117-128. doi:10.1080/01421590600622665
[25]	Watling, C., & Lingard, L. (2012). Toward meaningful evaluation of medical trainees: The influence of participants’ perceptions of the process. Advances in Health Science Education, 17, 183-194. doi:10.1007/s10459-010-9223-x