ExperienceDNA : A framework to conduct and analyse user tests in VR using the Wizard-of-Oz methodology
- Author
- Jamil Joundi (UGent) , Klaas Bombeke (UGent) , Niels Van Kets (UGent) , Wouter Durnez (UGent) , Jonas De Bruyne (UGent) , Glenn Van Wallendael (UGent) , Peter Lambert (UGent) , Jelle Saldien (UGent) and Lieven De Marez (UGent)
- Organization
- Abstract
- It is often challenging to measure participants’ reactions during user tests where a Wizard-of-Oz (WoZ) method is applied. This method is applied by the observers to validate functionalities of concepts that are difficult to build. In this study, a new technique is developed using virtual reality (VR) to improve the measurement of the participant’s reactions during such a test. In this system, called ExperienceDNA, VR user tests can be monitored and controlled through a desktop interface. In addition, physiological trackers (eye tracking and heart rate monitoring) are used to measure what the participant is looking at and to gauge their preferences. Moreover, the use of VR allows for quick adaptations to the virtual environment the participant is confronted with. In this way, highly versatile tests can be conducted while minimising the initial setup effort. Our approach has been validated by performing a pilot test on a predefined use case. The qualitative feedback collected from analysing batches of data from the pilot test is presented in the results section. In conclusion, this paper covers the development, description and evaluation of the ExperienceDNA framework, as well as some ideas for future improvements to this framework.
- Keywords
- Virtual reality, User testing, Design review, Virtual training, WoZ testing, HEART-RATE-VARIABILITY, VIRTUAL-REALITY, SYSTEMS
Downloads
-
ExperienceDNA HCII .pdf
- full text (Accepted manuscript)
- |
- open access
- |
- |
- 2.73 MB
-
(...).pdf
- full text (Published version)
- |
- UGent only
- |
- |
- 2.02 MB
Citation
Please use this url to cite or link to this publication: http://hdl.handle.net/1854/LU-8721071
- MLA
- Joundi, Jamil, et al. “ExperienceDNA : A Framework to Conduct and Analyse User Tests in VR Using the Wizard-of-Oz Methodology.” Design, User Experience, and Usability : Design for Contemporary Technological Environments : 10th International Conference, DUXU 2021, Held as Part of the 23rd HCI International Conference, HCII 2021 : Proceedings : Part III, edited by Marcelo M. Soares et al., vol. 12781, Springer, 2021, pp. 171–86, doi:10.1007/978-3-030-78227-6_13.
- APA
- Joundi, J., Bombeke, K., Van Kets, N., Durnez, W., De Bruyne, J., Van Wallendael, G., … De Marez, L. (2021). ExperienceDNA : A framework to conduct and analyse user tests in VR using the Wizard-of-Oz methodology. In M. M. Soares, E. Rosenzweig, & A. Marcus (Eds.), Design, user experience, and usability : design for contemporary technological environments : 10th International Conference, DUXU 2021, held as part of the 23rd HCI International Conference, HCII 2021 : Proceedings : Part III (Vol. 12781, pp. 171–186). https://doi.org/10.1007/978-3-030-78227-6_13
- Chicago author-date
- Joundi, Jamil, Klaas Bombeke, Niels Van Kets, Wouter Durnez, Jonas De Bruyne, Glenn Van Wallendael, Peter Lambert, Jelle Saldien, and Lieven De Marez. 2021. “ExperienceDNA : A Framework to Conduct and Analyse User Tests in VR Using the Wizard-of-Oz Methodology.” In Design, User Experience, and Usability : Design for Contemporary Technological Environments : 10th International Conference, DUXU 2021, Held as Part of the 23rd HCI International Conference, HCII 2021 : Proceedings : Part III, edited by Marcelo M. Soares, Elizabeth Rosenzweig, and Aaron Marcus, 12781:171–86. Springer. https://doi.org/10.1007/978-3-030-78227-6_13.
- Chicago author-date (all authors)
- Joundi, Jamil, Klaas Bombeke, Niels Van Kets, Wouter Durnez, Jonas De Bruyne, Glenn Van Wallendael, Peter Lambert, Jelle Saldien, and Lieven De Marez. 2021. “ExperienceDNA : A Framework to Conduct and Analyse User Tests in VR Using the Wizard-of-Oz Methodology.” In Design, User Experience, and Usability : Design for Contemporary Technological Environments : 10th International Conference, DUXU 2021, Held as Part of the 23rd HCI International Conference, HCII 2021 : Proceedings : Part III, ed by. Marcelo M. Soares, Elizabeth Rosenzweig, and Aaron Marcus, 12781:171–186. Springer. doi:10.1007/978-3-030-78227-6_13.
- Vancouver
- 1.Joundi J, Bombeke K, Van Kets N, Durnez W, De Bruyne J, Van Wallendael G, et al. ExperienceDNA : A framework to conduct and analyse user tests in VR using the Wizard-of-Oz methodology. In: Soares MM, Rosenzweig E, Marcus A, editors. Design, user experience, and usability : design for contemporary technological environments : 10th International Conference, DUXU 2021, held as part of the 23rd HCI International Conference, HCII 2021 : Proceedings : Part III. Springer; 2021. p. 171–86.
- IEEE
- [1]J. Joundi et al., “ExperienceDNA : A framework to conduct and analyse user tests in VR using the Wizard-of-Oz methodology,” in Design, user experience, and usability : design for contemporary technological environments : 10th International Conference, DUXU 2021, held as part of the 23rd HCI International Conference, HCII 2021 : Proceedings : Part III, Virtual event, 2021, vol. 12781, pp. 171–186.
@inproceedings{8721071,
abstract = {{It is often challenging to measure participants’ reactions during user tests where a Wizard-of-Oz (WoZ) method is applied. This method is applied by the observers to validate functionalities of concepts that are difficult to build. In this study, a new technique is developed using virtual reality (VR) to improve the measurement of the participant’s reactions during such a test. In this system, called ExperienceDNA, VR user tests can be monitored and controlled through a desktop interface. In addition, physiological trackers (eye tracking and heart rate monitoring) are used to measure what the participant is looking at and to gauge their preferences. Moreover, the use of VR allows for quick adaptations to the virtual environment the participant is confronted with. In this way, highly versatile tests can be conducted while minimising the initial setup effort. Our approach has been validated by performing a pilot test on a predefined use case. The qualitative feedback collected from analysing batches of data from the pilot test is presented in the results section. In conclusion, this paper covers the development, description and evaluation of the ExperienceDNA framework, as well as some ideas for future improvements to this framework.}},
author = {{Joundi, Jamil and Bombeke, Klaas and Van Kets, Niels and Durnez, Wouter and De Bruyne, Jonas and Van Wallendael, Glenn and Lambert, Peter and Saldien, Jelle and De Marez, Lieven}},
booktitle = {{Design, user experience, and usability : design for contemporary technological environments : 10th International Conference, DUXU 2021, held as part of the 23rd HCI International Conference, HCII 2021 : Proceedings : Part III}},
editor = {{Soares, Marcelo M. and Rosenzweig, Elizabeth and Marcus, Aaron}},
isbn = {{9783030782269}},
issn = {{0302-9743}},
keywords = {{Virtual reality,User testing,Design review,Virtual training,WoZ testing,HEART-RATE-VARIABILITY,VIRTUAL-REALITY,SYSTEMS}},
language = {{eng}},
location = {{Virtual event}},
pages = {{171--186}},
publisher = {{Springer}},
title = {{ExperienceDNA : A framework to conduct and analyse user tests in VR using the Wizard-of-Oz methodology}},
url = {{http://dx.doi.org/10.1007/978-3-030-78227-6_13}},
volume = {{12781}},
year = {{2021}},
}
- Altmetric
- View in Altmetric
- Web of Science
- Times cited: