Annotating topics, stance, argumentativeness and claims in Dutch social media comments : a pilot study
- Author
- Nina Bauwelinck and Els Lefever (UGent)
- Organization
- Project
- Abstract
- One of the major challenges currently facing the field of argumentation mining is the lack of consensus on how to analyse argumentative user-generated texts such as online comments. The theoretical motivations underlying the annotation guidelines used to generate labelled corpora rarely include motivation for the use of a particular theoretical basis. This pilot study reports on the annotation of a corpus of 100 Dutch user comments made in response to politically-themed news articles on Facebook. The annotation covers topic and aspect labelling, stance labelling, argumentativeness detection and claim identification. Our IAA study reports substantial agreement scores for argumentativeness detection (0.76 Fleiss’ kappa) and moderate agreement for claim labelling (0.45 Fleiss’ kappa). We provide a clear justification of the theories and definitions underlying the design of our guidelines. Our analysis of the annotations signal the importance of adjusting our guidelines to include allowances for missing context information and defining the concept of argumentativeness in connection with stance. Our annotated corpus and associated guidelines are made publicly available.
- Keywords
- LT3
Downloads
-
COLING2020 Argmining.pdf
- full text (Published version)
- |
- open access
- |
- |
- 177.26 KB
Citation
Please use this url to cite or link to this publication: http://hdl.handle.net/1854/LU-8682784
- MLA
- Bauwelinck, Nina, and Els Lefever. “Annotating Topics, Stance, Argumentativeness and Claims in Dutch Social Media Comments : A Pilot Study.” Proceedings of the 7th Workshop on Argument Mining (ArgMining 2020), edited by Elena Cabrio and Serena Villata, Association for Computational Linguistics (ACL), 2020, pp. 8–18.
- APA
- Bauwelinck, N., & Lefever, E. (2020). Annotating topics, stance, argumentativeness and claims in Dutch social media comments : a pilot study. In E. Cabrio & S. Villata (Eds.), Proceedings of the 7th Workshop on Argument Mining (ArgMining 2020) (pp. 8–18). Barcelona, Spain: Association for Computational Linguistics (ACL).
- Chicago author-date
- Bauwelinck, Nina, and Els Lefever. 2020. “Annotating Topics, Stance, Argumentativeness and Claims in Dutch Social Media Comments : A Pilot Study.” In Proceedings of the 7th Workshop on Argument Mining (ArgMining 2020), edited by Elena Cabrio and Serena Villata, 8–18. Barcelona, Spain: Association for Computational Linguistics (ACL).
- Chicago author-date (all authors)
- Bauwelinck, Nina, and Els Lefever. 2020. “Annotating Topics, Stance, Argumentativeness and Claims in Dutch Social Media Comments : A Pilot Study.” In Proceedings of the 7th Workshop on Argument Mining (ArgMining 2020), ed by. Elena Cabrio and Serena Villata, 8–18. Barcelona, Spain: Association for Computational Linguistics (ACL).
- Vancouver
- 1.Bauwelinck N, Lefever E. Annotating topics, stance, argumentativeness and claims in Dutch social media comments : a pilot study. In: Cabrio E, Villata S, editors. Proceedings of the 7th Workshop on Argument Mining (ArgMining 2020). Barcelona, Spain: Association for Computational Linguistics (ACL); 2020. p. 8–18.
- IEEE
- [1]N. Bauwelinck and E. Lefever, “Annotating topics, stance, argumentativeness and claims in Dutch social media comments : a pilot study,” in Proceedings of the 7th Workshop on Argument Mining (ArgMining 2020), Barcelona, Spain, 2020, pp. 8–18.
@inproceedings{8682784,
abstract = {{One of the major challenges currently facing the field of argumentation mining is the lack of
consensus on how to analyse argumentative user-generated texts such as online comments. The
theoretical motivations underlying the annotation guidelines used to generate labelled corpora
rarely include motivation for the use of a particular theoretical basis. This pilot study reports on
the annotation of a corpus of 100 Dutch user comments made in response to politically-themed
news articles on Facebook. The annotation covers topic and aspect labelling, stance labelling, argumentativeness detection and claim identification. Our IAA study reports substantial agreement
scores for argumentativeness detection (0.76 Fleiss’ kappa) and moderate agreement for claim
labelling (0.45 Fleiss’ kappa). We provide a clear justification of the theories and definitions
underlying the design of our guidelines. Our analysis of the annotations signal the importance of
adjusting our guidelines to include allowances for missing context information and defining the
concept of argumentativeness in connection with stance. Our annotated corpus and associated
guidelines are made publicly available.}},
author = {{Bauwelinck, Nina and Lefever, Els}},
booktitle = {{Proceedings of the 7th Workshop on Argument Mining (ArgMining 2020)}},
editor = {{Cabrio, Elena and Villata, Serena}},
isbn = {{9781952148446}},
keywords = {{LT3}},
language = {{eng}},
location = {{Barcelona, Spain}},
pages = {{8--18}},
publisher = {{Association for Computational Linguistics (ACL)}},
title = {{Annotating topics, stance, argumentativeness and claims in Dutch social media comments : a pilot study}},
url = {{https://www.aclweb.org/anthology/2020.argmining-1.2.pdf}},
year = {{2020}},
}