Advanced search
1 file | 424.05 KB Add to list

How to evaluate educational games: lessons learned from the evaluation study of Master FIND

Ellen Vanderhoven (UGent) , Bart Willems (UGent) , Stephanie Van Hove (UGent) , Anissa All (UGent) and Tammy Schellens (UGent)
Author
Organization
Abstract
Increasingly more researchers are emphasizing the importance of evaluating educational games in real life classroom settings, as most of these are developed without any theoretical consideration and without any evaluation afterwards. This causes uncertainty about the effectiveness of these games. Based on the reflections we made during a quasi-experimental evaluation study of MasterF.I.N.D., an educational game that aims to raise awareness about risks on social network sites, we want to put forth a few pitfalls that need to be taken into account when evaluating educational games. While we support the decision to evaluate such games using quasi-experimental designs in authentic classroom settings, we point towards the risks of evaluating games that were not developed based on theoretical considerations and practical requirements. We argue that including academic research only after development, might result in wasted effort and useless results. We illustrate this argument based on the case of the evaluation study of Master F.I.N.D., which was developed by industry without much theoretical consideration. Because of this lack of academic input during the development phase, we stumbled upon several problems that guided our research decisions and thereby jeopardized the validity of the research results. Based on this research example, we put forth some guidelines to take into account when starting an evaluation study. Furthermore, we propose another approach known in educational sciences as design-based research. This research approach involves the final users (i.e., pupils and teachers) during the full design process, and iteratively evaluates the impact of the game, resulting in both an evidence-based effective game and theoretical guidelines to develop such a game. We claim that this approach is necessary to establish both useful academic results and guidelines for practice.
Keywords
methodology, assessment, user involvement, design, theory, design-based research, DESIGN-BASED RESEARCH, PROGRESS

Downloads

  • (...).pdf
    • full text
    • |
    • UGent only
    • |
    • PDF
    • |
    • 424.05 KB

Citation

Please use this url to cite or link to this publication:

MLA
Vanderhoven, Ellen et al. “How to Evaluate Educational Games: Lessons Learned from the Evaluation Study of Master FIND.” Proceedings of the European Conference on Games-Based Learning. 2015. 548–553. Print.
APA
Vanderhoven, E., Willems, B., Van Hove, S., All, A., & Schellens, T. (2015). How to evaluate educational games: lessons learned from the evaluation study of Master FIND. Proceedings of the European Conference on Games-Based Learning (pp. 548–553). Presented at the 9th European Conference on Games-Based Learning (ECGBL).
Chicago author-date
Vanderhoven, Ellen, Bart Willems, Stephanie Van Hove, Anissa All, and Tammy Schellens. 2015. “How to Evaluate Educational Games: Lessons Learned from the Evaluation Study of Master FIND.” In Proceedings of the European Conference on Games-Based Learning, 548–553.
Chicago author-date (all authors)
Vanderhoven, Ellen, Bart Willems, Stephanie Van Hove, Anissa All, and Tammy Schellens. 2015. “How to Evaluate Educational Games: Lessons Learned from the Evaluation Study of Master FIND.” In Proceedings of the European Conference on Games-Based Learning, 548–553.
Vancouver
1.
Vanderhoven E, Willems B, Van Hove S, All A, Schellens T. How to evaluate educational games: lessons learned from the evaluation study of Master FIND. Proceedings of the European Conference on Games-Based Learning. 2015. p. 548–53.
IEEE
[1]
E. Vanderhoven, B. Willems, S. Van Hove, A. All, and T. Schellens, “How to evaluate educational games: lessons learned from the evaluation study of Master FIND,” in Proceedings of the European Conference on Games-Based Learning, Steinkjer, Norway, 2015, pp. 548–553.
@inproceedings{6924751,
  abstract     = {{Increasingly more researchers are emphasizing the importance of evaluating educational games in real life classroom settings, as most of these are developed without any theoretical consideration and without any evaluation afterwards. This causes uncertainty about the effectiveness of these games. Based on the reflections we made during a quasi-experimental evaluation study of MasterF.I.N.D., an educational game that aims to raise awareness about risks on social network sites, we want to put forth a few pitfalls that need to be taken into account when evaluating educational games. While we support the decision to evaluate such games using quasi-experimental designs in authentic classroom settings, we point towards the risks of evaluating games that were not developed based on theoretical considerations and practical requirements. We argue that including academic research only after development, might result in wasted effort and useless results. We illustrate this argument based on the case of the evaluation study of Master F.I.N.D., which was developed by industry without much theoretical consideration. Because of this lack of academic input during the development phase, we stumbled upon several problems that guided our research decisions and thereby jeopardized the validity of the research results. Based on this research example, we put forth some guidelines to take into account when starting an evaluation study. Furthermore, we propose another approach known in educational sciences as design-based research. This research approach involves the final users (i.e., pupils and teachers) during the full design process, and iteratively evaluates the impact of the game, resulting in both an evidence-based effective game and theoretical guidelines to develop such a game. We claim that this approach is necessary to establish both useful academic results and guidelines for practice.}},
  author       = {{Vanderhoven, Ellen and Willems, Bart and Van Hove, Stephanie and All, Anissa and Schellens, Tammy}},
  booktitle    = {{Proceedings of the European Conference on Games-Based Learning}},
  isbn         = {{978-1-910810-59-0}},
  issn         = {{2049-0992}},
  keywords     = {{methodology,assessment,user involvement,design,theory,design-based research,DESIGN-BASED RESEARCH,PROGRESS}},
  language     = {{eng}},
  location     = {{Steinkjer, Norway}},
  pages        = {{548--553}},
  title        = {{How to evaluate educational games: lessons learned from the evaluation study of Master FIND}},
  year         = {{2015}},
}

Web of Science
Times cited: