Advanced search
1 file | 223.34 KB

Using video-cases to assess student reflection : development and validation of an instrument

Author
Organization
Abstract
Background: Reflection is a meta-cognitive process, characterized by: 1. Awareness of self and the situation; 2. Critical analysis and understanding of both self and the situation; 3. Development of new perspectives to inform future actions. Assessors can only access reflections indirectly through learners’ verbal and/or written expressions. Being privy to the situation that triggered reflection could place reflective materials into context. Video-cases make that possible and, coupled with a scoring rubric, offer a reliable way of assessing reflection. Methods: Fourth and fifth year undergraduate medical students were shown two interactive video-cases and asked to reflect on this experience, guided by six standard questions. The quality of students’ reflections were scored using a specially developed Student Assessment of Reflection Scoring rubric (StARS®). Reflection scores were analyzed concerning interrater reliability and ability to discriminate between students. Further, the intra-rater reliability and case specificity were estimated by means of a generalizability study with rating and case scenario as facets. Results: Reflection scores of 270 students ranged widely and interrater reliability was acceptable (Krippendorff’s alpha = 0.88). The generalizability study suggested 3 or 4 cases were needed to obtain reliable ratings from 4th year students and ≥ 6 cases from 5th year students. Conclusion : Use of StARS® to assess student reflections triggered by standardized video-cases had acceptable discriminative ability and reliability. We offer this practical method for assessing reflection summatively, and providing formative feedback in training situations.
Keywords
MEDICAL COMMUNICATION-SKILLS, assessment, reflection, undergraduate medical education, METACOGNITIVE AWARENESS, GENERALIZABILITY, RELIABILITY, TEACHERS, VALIDITY, FEEDBACK, THINKING, RUBRICS, LEVEL

Downloads

  • Koole BME ip.pdf
    • full text
    • |
    • open access
    • |
    • PDF
    • |
    • 223.34 KB

Citation

Please use this url to cite or link to this publication:

Chicago
Koole, Sebastiaan, Tim Dornan, Leen Aper, Bram De Wever, Albert Scherpbier, Martin Valcke, Janke Cohen-Schotanus, and Anselme Derese. 2012. “Using Video-cases to Assess Student Reflection : Development and Validation of an Instrument.” Bmc Medical Education 12.
APA
Koole, S., Dornan, T., Aper, L., De Wever, B., Scherpbier, A., Valcke, M., Cohen-Schotanus, J., et al. (2012). Using video-cases to assess student reflection : development and validation of an instrument. BMC MEDICAL EDUCATION, 12.
Vancouver
1.
Koole S, Dornan T, Aper L, De Wever B, Scherpbier A, Valcke M, et al. Using video-cases to assess student reflection : development and validation of an instrument. BMC MEDICAL EDUCATION. 2012;12.
MLA
Koole, Sebastiaan, Tim Dornan, Leen Aper, et al. “Using Video-cases to Assess Student Reflection : Development and Validation of an Instrument.” BMC MEDICAL EDUCATION 12 (2012): n. pag. Print.
@article{2095093,
  abstract     = {Background: Reflection is a meta-cognitive process, characterized by: 1. Awareness of self and the situation; 2. Critical analysis and understanding of both self and the situation; 3. Development of new perspectives to inform future actions. Assessors can only access reflections indirectly through learners{\textquoteright} verbal and/or written expressions. Being privy to the situation that triggered reflection could place reflective materials into context. Video-cases make that possible and, coupled with a scoring rubric, offer a reliable way of assessing reflection.
Methods: Fourth and fifth year undergraduate medical students were shown two interactive video-cases and asked to reflect on this experience, guided by six standard questions. The quality of students{\textquoteright} reflections were scored using a specially developed Student Assessment of Reflection Scoring rubric (StARS{\textregistered}). Reflection scores were analyzed concerning interrater reliability and ability to discriminate between students. Further, the intra-rater reliability and case specificity were estimated by means of a generalizability study with rating and case scenario as facets.
Results: Reflection scores of 270 students ranged widely and interrater reliability was acceptable (Krippendorff{\textquoteright}s alpha = 0.88). The generalizability study suggested 3 or 4 cases were needed to obtain reliable ratings from 4th year students and \ensuremath{\geq} 6 cases from 5th year students.
Conclusion : Use of StARS{\textregistered} to assess student reflections triggered by standardized video-cases had acceptable discriminative ability and reliability. We offer this practical method for assessing reflection summatively, and providing formative feedback in training situations.},
  articleno    = {22},
  author       = {Koole, Sebastiaan and Dornan, Tim and Aper, Leen and De Wever, Bram and Scherpbier, Albert and Valcke, Martin and Cohen-Schotanus, Janke and Derese, Anselme},
  issn         = {1472-6920},
  journal      = {BMC MEDICAL EDUCATION},
  keyword      = {MEDICAL COMMUNICATION-SKILLS,assessment,reflection,undergraduate medical education,METACOGNITIVE AWARENESS,GENERALIZABILITY,RELIABILITY,TEACHERS,VALIDITY,FEEDBACK,THINKING,RUBRICS,LEVEL},
  language     = {eng},
  pages        = {8},
  title        = {Using video-cases to assess student reflection : development and validation of an instrument},
  url          = {http://dx.doi.org/10.1186/1472-6920-12-22},
  volume       = {12},
  year         = {2012},
}

Altmetric
View in Altmetric
Web of Science
Times cited: