Pre-test session impact on the effectiveness assessment of a fire safety game
- Author
- Anissa All (UGent) , Barbara Plovie, Elena Patricia Nunez Castellar (UGent) and Jan Van Looy (UGent)
- Organization
- Abstract
- In recent years, critiques have been formulated regarding current evaluation methods of DGBL (digital game-based learning) effectiveness, putting the validity of certain results in doubt. An important point of discussion in DGBL effectiveness studies is whether or not a pre-test should be administered, as it can lead to practice effects and pre-test sensitization, threatening internal validity of the results. The present study aims at testing if the administration of a pre-test has a direct influence on post-test scores and/or makes participants more receptive to the intervention. For this purpose, an effectiveness study of a fire safety training in a hospital was conducted using a Solomon four-group design. The experimental groups received a game-based intervention (n= 65) of which 34 participants received a pre-test and 31 did not. The control groups received traditional classroom instruction (n=68), of which 39 participants received a pre-test and 29 did not. A 2x2 ANOVA was used to explore the practice effect and the interaction between the pre-test and the intervention. An interaction effect between pre-test and intervention is detected. More specifically, this interaction takes place in the traditional classroom group, indicating pre-test sensitization. In the traditional classroom context, the pre-test makes the participants more sensitive to the content treated in the intervention while administration of a pre-test does not influence outcomes of the DGBL treatment. When the administration of a pre-test influences the control group's receptivity to the treatment, but not the experimental group, results of an effectiveness study may be biased. This is especially relevant in the DGBL field as often, non-significant differences between DGBL and more traditional methods are reported. Therefore, further research should take this into account and look for possible solutions to solve this discrepancy.
- Keywords
- EXTENSION, DESIGN, Solomon 4-group design, DGBL, fire safety, pre-test sensitization, practice effect
Downloads
-
ECGBL2015 Pre-test session impact.pdf
- full text
- |
- open access
- |
- |
- 388.50 KB
Citation
Please use this url to cite or link to this publication: http://hdl.handle.net/1854/LU-8136407
- MLA
- All, Anissa, et al. “Pre-Test Session Impact on the Effectiveness Assessment of a Fire Safety Game.” Proceedings of the European Conference on Games-Based Learning, 2015, pp. 623–30.
- APA
- All, A., Plovie, B., Nunez Castellar, E. P., & Van Looy, J. (2015). Pre-test session impact on the effectiveness assessment of a fire safety game. Proceedings of the European Conference on Games-Based Learning, 623–630.
- Chicago author-date
- All, Anissa, Barbara Plovie, Elena Patricia Nunez Castellar, and Jan Van Looy. 2015. “Pre-Test Session Impact on the Effectiveness Assessment of a Fire Safety Game.” In Proceedings of the European Conference on Games-Based Learning, 623–30.
- Chicago author-date (all authors)
- All, Anissa, Barbara Plovie, Elena Patricia Nunez Castellar, and Jan Van Looy. 2015. “Pre-Test Session Impact on the Effectiveness Assessment of a Fire Safety Game.” In Proceedings of the European Conference on Games-Based Learning, 623–630.
- Vancouver
- 1.All A, Plovie B, Nunez Castellar EP, Van Looy J. Pre-test session impact on the effectiveness assessment of a fire safety game. In: Proceedings of the European Conference on Games-Based Learning. 2015. p. 623–30.
- IEEE
- [1]A. All, B. Plovie, E. P. Nunez Castellar, and J. Van Looy, “Pre-test session impact on the effectiveness assessment of a fire safety game,” in Proceedings of the European Conference on Games-Based Learning, Steinkjer, Norway, 2015, pp. 623–630.
@inproceedings{8136407, abstract = {{In recent years, critiques have been formulated regarding current evaluation methods of DGBL (digital game-based learning) effectiveness, putting the validity of certain results in doubt. An important point of discussion in DGBL effectiveness studies is whether or not a pre-test should be administered, as it can lead to practice effects and pre-test sensitization, threatening internal validity of the results. The present study aims at testing if the administration of a pre-test has a direct influence on post-test scores and/or makes participants more receptive to the intervention. For this purpose, an effectiveness study of a fire safety training in a hospital was conducted using a Solomon four-group design. The experimental groups received a game-based intervention (n= 65) of which 34 participants received a pre-test and 31 did not. The control groups received traditional classroom instruction (n=68), of which 39 participants received a pre-test and 29 did not. A 2x2 ANOVA was used to explore the practice effect and the interaction between the pre-test and the intervention. An interaction effect between pre-test and intervention is detected. More specifically, this interaction takes place in the traditional classroom group, indicating pre-test sensitization. In the traditional classroom context, the pre-test makes the participants more sensitive to the content treated in the intervention while administration of a pre-test does not influence outcomes of the DGBL treatment. When the administration of a pre-test influences the control group's receptivity to the treatment, but not the experimental group, results of an effectiveness study may be biased. This is especially relevant in the DGBL field as often, non-significant differences between DGBL and more traditional methods are reported. Therefore, further research should take this into account and look for possible solutions to solve this discrepancy.}}, author = {{All, Anissa and Plovie, Barbara and Nunez Castellar, Elena Patricia and Van Looy, Jan}}, booktitle = {{Proceedings of the European Conference on Games-Based Learning}}, isbn = {{978-1-910810-59-0}}, issn = {{2049-0992}}, keywords = {{EXTENSION,DESIGN,Solomon 4-group design,DGBL,fire safety,pre-test sensitization,practice effect}}, language = {{eng}}, location = {{Steinkjer, Norway}}, pages = {{623--630}}, title = {{Pre-test session impact on the effectiveness assessment of a fire safety game}}, url = {{https://apps.webofknowledge.com/full_record.do?product=WOS&search_mode=GeneralSearch&qid=6&SID=N26yMgDB8sBoNOMkERF&page=1&doc=1}}, year = {{2015}}, }