Academic Bibliography

 Search 200 years of publications by Ghent University researchers.
Advanced search
1 file | 1.92 MB
Add to list
  1. Search results
  2. Variation in the expression and annot...

Variation in the expression and annotation of emotions : a Wizard of Oz pilot study

Sofie Labat (UGent) , Naomi Ackaert, Thomas Demeester (UGent) and Veronique Hoste (UGent)
(2022) Proceedings of the 1st Workshop on Perspectivist Approaches to NLP @LREC2022. p.66-72
Author
Organization
Project
Abstract
This pilot study employs the Wizard of Oz technique to collect a corpus of written human-computer conversations in the domain of customer service. The resulting dataset contains 192 conversations and is used to test three hypotheses related to the expression and annotation of emotions. First, we hypothesize that there is a discrepancy between the emotion annotations of the participant (the experiencer) and the annotations of our external annotator (the observer). Furthermore, we hypothesize that the personality of the participants has an influence on the emotions they expressed, and on the way they evaluated (annotated) these emotions. We found that for an external, trained annotator, not all emotion labels were equally easy to work with. We also noticed that the trained annotator had a tendency to opt for emotion labels that were more centered in the valence-arousal space, while participants made more `extreme' annotations. For the second hypothesis, we discovered a positive correlation between the personality trait extraversion and the emotion dimensions valence and dominance in our sample. Finally, for the third premise, we observed a positive correlation between the internal-external agreement on emotion labels and the personality traits conscientiousness and extraversion. Our insights and findings will be used in future research to conduct a larger Wizard of Oz experiment.
Keywords
emotion analysis, Wizard of Oz study, conversational data collection, customer profiling, customer service, LT3

Downloads

  • Download
    2022.nlperspectives-1.9.pdf
    • full text (Published version)
    • |
    • open access
    • |
    • PDF
    • |
    • 1.92 MB

Citation

Please use this url to cite or link to this publication:

MLA
Labat, Sofie, et al. “Variation in the Expression and Annotation of Emotions : A Wizard of Oz Pilot Study.” Proceedings of the 1st Workshop on Perspectivist Approaches to NLP @LREC2022, edited by Gavin Abercrombie et al., European Language Resources Association (ELRA), 2022, pp. 66–72.
APA
Labat, S., Ackaert, N., Demeester, T., & Hoste, V. (2022). Variation in the expression and annotation of emotions : a Wizard of Oz pilot study. In G. Abercrombie, V. Basile, S. Tonelli, V. Rieser, & A. Uma (Eds.), Proceedings of the 1st Workshop on Perspectivist Approaches to NLP @LREC2022 (pp. 66–72). Marseille, France: European Language Resources Association (ELRA).
Chicago author-date
Labat, Sofie, Naomi Ackaert, Thomas Demeester, and Veronique Hoste. 2022. “Variation in the Expression and Annotation of Emotions : A Wizard of Oz Pilot Study.” In Proceedings of the 1st Workshop on Perspectivist Approaches to NLP @LREC2022, edited by Gavin Abercrombie, Valerio Basile, Sara Tonelli, Verena Rieser, and Alexandra Uma, 66–72. Marseille, France: European Language Resources Association (ELRA).
Chicago author-date (all authors)
Labat, Sofie, Naomi Ackaert, Thomas Demeester, and Veronique Hoste. 2022. “Variation in the Expression and Annotation of Emotions : A Wizard of Oz Pilot Study.” In Proceedings of the 1st Workshop on Perspectivist Approaches to NLP @LREC2022, ed by. Gavin Abercrombie, Valerio Basile, Sara Tonelli, Verena Rieser, and Alexandra Uma, 66–72. Marseille, France: European Language Resources Association (ELRA).
Vancouver
1.
Labat S, Ackaert N, Demeester T, Hoste V. Variation in the expression and annotation of emotions : a Wizard of Oz pilot study. In: Abercrombie G, Basile V, Tonelli S, Rieser V, Uma A, editors. Proceedings of the 1st Workshop on Perspectivist Approaches to NLP @LREC2022. Marseille, France: European Language Resources Association (ELRA); 2022. p. 66–72.
IEEE
[1]
S. Labat, N. Ackaert, T. Demeester, and V. Hoste, “Variation in the expression and annotation of emotions : a Wizard of Oz pilot study,” in Proceedings of the 1st Workshop on Perspectivist Approaches to NLP @LREC2022, Marseille, France, 2022, pp. 66–72.
@inproceedings{8756832,
  abstract     = {{This pilot study employs the Wizard of Oz technique to collect a corpus of written human-computer conversations in the domain of customer service. The resulting dataset contains 192 conversations and is used to test three hypotheses related to the expression and annotation of emotions. First, we hypothesize that there is a discrepancy between the emotion annotations of the participant (the experiencer) and the annotations of our external annotator (the observer). Furthermore, we hypothesize that the personality of the participants has an influence on the emotions they expressed, and on the way they evaluated (annotated) these emotions. We found that for an external, trained annotator, not all emotion labels were equally easy to work with. We also noticed that the trained annotator had a tendency to opt for emotion labels that were more centered in the valence-arousal space, while participants made more `extreme' annotations. For the second hypothesis, we discovered a positive correlation between the personality trait extraversion and the emotion dimensions valence and dominance in our sample. Finally, for the third premise, we observed a positive correlation between the internal-external agreement on emotion labels and the personality traits conscientiousness and extraversion. Our insights and findings will be used in future research to conduct a larger Wizard of Oz experiment.}},
  author       = {{Labat, Sofie and Ackaert, Naomi and Demeester, Thomas and Hoste, Veronique}},
  booktitle    = {{Proceedings of the 1st Workshop on Perspectivist Approaches to NLP @LREC2022}},
  editor       = {{Abercrombie, Gavin and Basile, Valerio and Tonelli, Sara and Rieser, Verena and Uma, Alexandra}},
  isbn         = {{9791095546986}},
  keywords     = {{emotion analysis,Wizard of Oz study,conversational data collection,customer profiling,customer service,LT3}},
  language     = {{eng}},
  location     = {{Marseille, France}},
  pages        = {{66--72}},
  publisher    = {{European Language Resources Association (ELRA)}},
  title        = {{Variation in the expression and annotation of emotions : a Wizard of Oz pilot study}},
  url          = {{http://www.lrec-conf.org/proceedings/lrec2022/workshops/NLPerspectives/index.html}},
  year         = {{2022}},
}