A systematic literature review of methodology used to measure effectiveness in digital game-based learning
- Author
- Anissa All (UGent) , Elena Patricia Nunez Castellar (UGent) and Jan Van Looy (UGent)
- Organization
- Abstract
- In recent years, a growing number of studies is being conducted into the effectiveness of digital game-based learning (DGBL). Despite this growing interest, however, it remains difficult to draw general conclusions due to the disparities in methods and reporting. Guidelines or a standardized procedure for conducting DGBL effectiveness research would allow to compare results across studies and provide well-founded and more generalizable evidence for the impact of DGBL. This study presents a first step in this process by mapping current practices through a systematic literature review. The review included peer-reviewed journal and conference publications between 2000 and 2012. Other inclusion criteria were that (1) the study’s primary aim was effectiveness measurement of cognitive learning outcomes, (2) the focus was on digital games and (3) a pre-post design with a control group was used. Twenty-five publications were found eligible for this study. Important differences were found in the number of control groups used and the type of intervention implemented in the control group (e.g. traditional classroom teaching, use of multimedia, computer-based learning, paper exercises, other games, or no intervention). Regarding the implementation method of the DGBL intervention in the experimental group, two approaches can be distinguished: stand-alone intervention or as part of a larger program. Moreover, a wide variety of effectiveness measures was used: measures for learning outcomes were complemented with time measurements and/or with self-reported measurements for self-efficacy and motivation. Learning effect calculation also varied, introducing pre-test scores in the analysis, conducting a separate analysis on pre- and post-test scores or conducting an analysis on difference scores. Our study thus indicates that a variety of methods is being used in DGBL effectiveness research opening a discussion regarding the potential and requirements for future procedural guidelines.
- Keywords
- digital game-based learning, effectiveness, REAL, cognitive learning outcomes
Downloads
-
ECGBL All NunezCastellar VanLooy.pdf
- full text
- |
- open access
- |
- |
- 633.41 KB
Citation
Please use this url to cite or link to this publication: http://hdl.handle.net/1854/LU-4157899
- MLA
- All, Anissa, et al. “A Systematic Literature Review of Methodology Used to Measure Effectiveness in Digital Game-Based Learning.” Proceedings of the European Conference on Games-Based Learning, edited by P Escudeiro and CV DeCarvalho, Academic Conferences, Publishing International Limited, 2013, pp. 607–16.
- APA
- All, A., Nunez Castellar, E. P., & Van Looy, J. (2013). A systematic literature review of methodology used to measure effectiveness in digital game-based learning. In P. Escudeiro & C. DeCarvalho (Eds.), Proceedings of the European Conference on Games-Based Learning (pp. 607–616). Reading, UK: Academic Conferences, Publishing International Limited.
- Chicago author-date
- All, Anissa, Elena Patricia Nunez Castellar, and Jan Van Looy. 2013. “A Systematic Literature Review of Methodology Used to Measure Effectiveness in Digital Game-Based Learning.” In Proceedings of the European Conference on Games-Based Learning, edited by P Escudeiro and CV DeCarvalho, 607–16. Reading, UK: Academic Conferences, Publishing International Limited.
- Chicago author-date (all authors)
- All, Anissa, Elena Patricia Nunez Castellar, and Jan Van Looy. 2013. “A Systematic Literature Review of Methodology Used to Measure Effectiveness in Digital Game-Based Learning.” In Proceedings of the European Conference on Games-Based Learning, ed by. P Escudeiro and CV DeCarvalho, 607–616. Reading, UK: Academic Conferences, Publishing International Limited.
- Vancouver
- 1.All A, Nunez Castellar EP, Van Looy J. A systematic literature review of methodology used to measure effectiveness in digital game-based learning. In: Escudeiro P, DeCarvalho C, editors. Proceedings of the European Conference on Games-Based Learning. Reading, UK: Academic Conferences, Publishing International Limited; 2013. p. 607–16.
- IEEE
- [1]A. All, E. P. Nunez Castellar, and J. Van Looy, “A systematic literature review of methodology used to measure effectiveness in digital game-based learning,” in Proceedings of the European Conference on Games-Based Learning, Porto, Portugal, 2013, pp. 607–616.
@inproceedings{4157899,
abstract = {{In recent years, a growing number of studies is being conducted into the effectiveness of digital game-based learning (DGBL). Despite this growing interest, however, it remains difficult to draw general conclusions due to the disparities in methods and reporting. Guidelines or a standardized procedure for conducting DGBL effectiveness research would allow to compare results across studies and provide well-founded and more generalizable evidence for the impact of DGBL. This study presents a first step in this process by mapping current practices through a systematic literature review. The review included peer-reviewed journal and conference publications between 2000 and 2012. Other inclusion criteria were that (1) the study’s primary aim was effectiveness measurement of cognitive learning outcomes, (2) the focus was on digital games and (3) a pre-post design with a control group was used. Twenty-five publications were found eligible for this study. Important differences were found in the number of control groups used and the type of intervention implemented in the control group (e.g. traditional classroom teaching, use of multimedia, computer-based learning, paper exercises, other games, or no intervention). Regarding the implementation method of the DGBL intervention in the experimental group, two approaches can be distinguished: stand-alone intervention or as part of a larger program. Moreover, a wide variety of effectiveness measures was used: measures for learning outcomes were complemented with time measurements and/or with self-reported measurements for self-efficacy and motivation. Learning effect calculation also varied, introducing pre-test scores in the analysis, conducting a separate analysis on pre- and post-test scores or conducting an analysis on difference scores. Our study thus indicates that a variety of methods is being used in DGBL effectiveness research opening a discussion regarding the potential and requirements for future procedural guidelines.}},
author = {{All, Anissa and Nunez Castellar, Elena Patricia and Van Looy, Jan}},
booktitle = {{Proceedings of the European Conference on Games-Based Learning}},
editor = {{Escudeiro, P and DeCarvalho, CV}},
isbn = {{9781909507654}},
issn = {{2049-0992}},
keywords = {{digital game-based learning,effectiveness,REAL,cognitive learning outcomes}},
language = {{eng}},
location = {{Porto, Portugal}},
pages = {{607--616}},
publisher = {{Academic Conferences, Publishing International Limited}},
title = {{A systematic literature review of methodology used to measure effectiveness in digital game-based learning}},
year = {{2013}},
}