Ghent University Academic Bibliography

Advanced

Using video-cases to assess student reflection: development and validation of an instrument

Sebastiaan Koole UGent, Tim Dornan, Leen Aper UGent, Bram De Wever UGent, Albert Scherpbier, Martin Valcke UGent, Janke Cohen-Schotanus and Anselme Derese UGent (2012) BMC MEDICAL EDUCATION. 12.
abstract
Background: Reflection is a meta-cognitive process, characterized by: 1. Awareness of self and the situation; 2. Critical analysis and understanding of both self and the situation; 3. Development of new perspectives to inform future actions. Assessors can only access reflections indirectly through learners’ verbal and/or written expressions. Being privy to the situation that triggered reflection could place reflective materials into context. Video-cases make that possible and, coupled with a scoring rubric, offer a reliable way of assessing reflection. Methods: Fourth and fifth year undergraduate medical students were shown two interactive video-cases and asked to reflect on this experience, guided by six standard questions. The quality of students’ reflections were scored using a specially developed Student Assessment of Reflection Scoring rubric (StARS®). Reflection scores were analyzed concerning interrater reliability and ability to discriminate between students. Further, the intra-rater reliability and case specificity were estimated by means of a generalizability study with rating and case scenario as facets. Results: Reflection scores of 270 students ranged widely and interrater reliability was acceptable (Krippendorff’s alpha = 0.88). The generalizability study suggested 3 or 4 cases were needed to obtain reliable ratings from 4th year students and ≥ 6 cases from 5th year students. Conclusion : Use of StARS® to assess student reflections triggered by standardized video-cases had acceptable discriminative ability and reliability. We offer this practical method for assessing reflection summatively, and providing formative feedback in training situations.
Please use this url to cite or link to this publication:
author
organization
year
type
journalArticle (original)
publication status
published
subject
keyword
MEDICAL COMMUNICATION-SKILLS, assessment, reflection, undergraduate medical education, METACOGNITIVE AWARENESS, GENERALIZABILITY, RELIABILITY, TEACHERS, VALIDITY, FEEDBACK, THINKING, RUBRICS, LEVEL
journal title
BMC MEDICAL EDUCATION
BMC Med. Educ.
volume
12
article_number
22
pages
8 pages
Web of Science type
Article
Web of Science id
000307959500001
JCR category
EDUCATION & EDUCATIONAL RESEARCH
JCR impact factor
1.409 (2012)
JCR rank
31/215 (2012)
JCR quartile
1 (2012)
ISSN
1472-6920
DOI
10.1186/1472-6920-12-22
language
English
UGent publication?
yes
classification
A1
copyright statement
I have retained and own the full copyright for this publication
id
2095093
handle
http://hdl.handle.net/1854/LU-2095093
date created
2012-04-26 14:09:04
date last changed
2015-06-17 10:01:41
@article{2095093,
  abstract     = {Background: Reflection is a meta-cognitive process, characterized by: 1. Awareness of self and the situation; 2. Critical analysis and understanding of both self and the situation; 3. Development of new perspectives to inform future actions. Assessors can only access reflections indirectly through learners{\textquoteright} verbal and/or written expressions. Being privy to the situation that triggered reflection could place reflective materials into context. Video-cases make that possible and, coupled with a scoring rubric, offer a reliable way of assessing reflection.
Methods: Fourth and fifth year undergraduate medical students were shown two interactive video-cases and asked to reflect on this experience, guided by six standard questions. The quality of students{\textquoteright} reflections were scored using a specially developed Student Assessment of Reflection Scoring rubric (StARS{\textregistered}). Reflection scores were analyzed concerning interrater reliability and ability to discriminate between students. Further, the intra-rater reliability and case specificity were estimated by means of a generalizability study with rating and case scenario as facets.
Results: Reflection scores of 270 students ranged widely and interrater reliability was acceptable (Krippendorff{\textquoteright}s alpha = 0.88). The generalizability study suggested 3 or 4 cases were needed to obtain reliable ratings from 4th year students and \ensuremath{\geq} 6 cases from 5th year students.
Conclusion : Use of StARS{\textregistered} to assess student reflections triggered by standardized video-cases had acceptable discriminative ability and reliability. We offer this practical method for assessing reflection summatively, and providing formative feedback in training situations.},
  articleno    = {22},
  author       = {Koole, Sebastiaan and Dornan, Tim and Aper, Leen and De Wever, Bram and Scherpbier, Albert and Valcke, Martin and Cohen-Schotanus, Janke and Derese, Anselme},
  issn         = {1472-6920},
  journal      = {BMC MEDICAL EDUCATION},
  keyword      = {MEDICAL COMMUNICATION-SKILLS,assessment,reflection,undergraduate medical education,METACOGNITIVE AWARENESS,GENERALIZABILITY,RELIABILITY,TEACHERS,VALIDITY,FEEDBACK,THINKING,RUBRICS,LEVEL},
  language     = {eng},
  pages        = {8},
  title        = {Using video-cases to assess student reflection: development and validation of an instrument},
  url          = {http://dx.doi.org/10.1186/1472-6920-12-22},
  volume       = {12},
  year         = {2012},
}

Chicago
Koole, Sebastiaan, Tim Dornan, Leen Aper, Bram De Wever, Albert Scherpbier, Martin Valcke, Janke Cohen-Schotanus, and Anselme Derese. 2012. “Using Video-cases to Assess Student Reflection: Development and Validation of an Instrument.” Bmc Medical Education 12.
APA
Koole, S., Dornan, T., Aper, L., De Wever, B., Scherpbier, A., Valcke, M., Cohen-Schotanus, J., et al. (2012). Using video-cases to assess student reflection: development and validation of an instrument. BMC MEDICAL EDUCATION, 12.
Vancouver
1.
Koole S, Dornan T, Aper L, De Wever B, Scherpbier A, Valcke M, et al. Using video-cases to assess student reflection: development and validation of an instrument. BMC MEDICAL EDUCATION. 2012;12.
MLA
Koole, Sebastiaan, Tim Dornan, Leen Aper, et al. “Using Video-cases to Assess Student Reflection: Development and Validation of an Instrument.” BMC MEDICAL EDUCATION 12 (2012): n. pag. Print.