SemEval-2018 task 3 : irony detection in English tweets
- Author
- Cynthia Van Hee (UGent) , Els Lefever (UGent) and Veronique Hoste (UGent)
- Organization
- Abstract
- This paper presents the first shared task on irony detection: given a tweet, automatic natural language processing systems should determine whether the tweet is ironic (Task A) and which type of irony (if any) is expressed (Task B). The ironic tweets were collected using irony-related hashtags (i.e. #irony, #sarcasm, #not) and were subsequently manually annotated to minimise the amount of noise in the corpus. Prior to distributing the data, hashtags that were used to collect the tweets were removed from the corpus. For both tasks, a training corpus of 3,834 tweets was provided, as well as a test set containing 784 tweets. Our shared tasks received submissions from 43 teams for the binary classification Task A and from 31 teams for the multiclass Task B. The highest classification scores obtained for both subtasks are respectively F1= 0.71 and F1= 0.51 and demonstrate that fine-grained irony classification is much more challenging than binary irony detection.
- Keywords
- irony detection, lt3
Downloads
-
(...).pdf
- full text
- |
- UGent only
- |
- |
- 196.25 KB
Citation
Please use this url to cite or link to this publication: http://hdl.handle.net/1854/LU-8567157
- MLA
- Van Hee, Cynthia, et al. “SemEval-2018 Task 3 : Irony Detection in English Tweets.” Proceedings of The 12th International Workshop on Semantic Evaluation, Association for Computational Linguistics, 2018, pp. 39–50.
- APA
- Van Hee, C., Lefever, E., & Hoste, V. (2018). SemEval-2018 task 3 : irony detection in English tweets. Proceedings of The 12th International Workshop on Semantic Evaluation, 39–50. New Orleans, Louisiana: Association for Computational Linguistics.
- Chicago author-date
- Van Hee, Cynthia, Els Lefever, and Veronique Hoste. 2018. “SemEval-2018 Task 3 : Irony Detection in English Tweets.” In Proceedings of The 12th International Workshop on Semantic Evaluation, 39–50. New Orleans, Louisiana: Association for Computational Linguistics.
- Chicago author-date (all authors)
- Van Hee, Cynthia, Els Lefever, and Veronique Hoste. 2018. “SemEval-2018 Task 3 : Irony Detection in English Tweets.” In Proceedings of The 12th International Workshop on Semantic Evaluation, 39–50. New Orleans, Louisiana: Association for Computational Linguistics.
- Vancouver
- 1.Van Hee C, Lefever E, Hoste V. SemEval-2018 task 3 : irony detection in English tweets. In: Proceedings of The 12th International Workshop on Semantic Evaluation. New Orleans, Louisiana: Association for Computational Linguistics; 2018. p. 39–50.
- IEEE
- [1]C. Van Hee, E. Lefever, and V. Hoste, “SemEval-2018 task 3 : irony detection in English tweets,” in Proceedings of The 12th International Workshop on Semantic Evaluation, New Orleans, Louisiana, 2018, pp. 39–50.
@inproceedings{8567157,
abstract = {{This paper presents the first shared task on irony detection: given a tweet, automatic natural language processing systems should determine whether the tweet is ironic (Task A) and which type of irony (if any) is expressed (Task B). The ironic tweets were collected using irony-related hashtags (i.e. #irony, #sarcasm, #not) and were subsequently manually annotated to minimise the amount of noise in the corpus. Prior to distributing the data, hashtags that were used to collect the tweets were removed from the corpus. For both tasks, a training corpus of 3,834 tweets was provided, as well as a test set containing 784 tweets. Our shared tasks received submissions from 43 teams for the binary classification Task A and from 31 teams for the multiclass Task B. The highest classification scores obtained for both subtasks are respectively F1= 0.71 and F1= 0.51 and demonstrate that fine-grained irony classification is much more challenging than binary irony detection.}},
author = {{Van Hee, Cynthia and Lefever, Els and Hoste, Veronique}},
booktitle = {{Proceedings of The 12th International Workshop on Semantic Evaluation}},
isbn = {{9781948087209}},
keywords = {{irony detection,lt3}},
language = {{eng}},
location = {{New Orleans, Louisiana}},
pages = {{39--50}},
publisher = {{Association for Computational Linguistics}},
title = {{SemEval-2018 task 3 : irony detection in English tweets}},
url = {{http://aclweb.org/anthology/S18-1005}},
year = {{2018}},
}