Integrating meaning into quality evaluation of machine translation
- Author
- Osman Başkaya, Eray Yildiz, Doruk Tunaoğlu, Mustafa Tolga Eren and A. Seza Doğruöz (UGent)
- Organization
- Abstract
- Machine translation (MT) quality is evaluated through comparisons between MT outputs and the human translations (HT). Traditionally, this evaluation relies on form related features (e.g. lexicon and syntax) and ignores the transfer of meaning reflected in HT outputs. Instead, we evaluate the quality of MT outputs through meaning related features (e.g. polarity, subjectivity) with two experiments. In the first experiment, the meaning related features are compared to human rankings individually. In the second experiment, combinations of meaning related features and other quality metrics are utilized to predict the same human rankings. The results of our experiments confirm the benefit of these features in predicting human evaluation of translation quality in addition to traditional metrics which focus mainly on form.
- Keywords
- Machine Translation quality, form and meaning in Machine Translation, MT quality metrics, lt3
Downloads
-
Dogruoz EACL 2017 MachineTranslationQuality.pdf
- full text (Published version)
- |
- open access
- |
- |
- 220.01 KB
Citation
Please use this url to cite or link to this publication: http://hdl.handle.net/1854/LU-8694738
- MLA
- Başkaya, Osman, et al. “Integrating Meaning into Quality Evaluation of Machine Translation.” 15TH CONFERENCE OF THE EUROPEAN CHAPTER OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS (EACL 2017), VOL 1: LONG PAPERS, Association for Computational Linguistics (ACL), 2017, pp. 210–19.
- APA
- Başkaya, O., Yildiz, E., Tunaoğlu, D., Eren, M. T., & Doğruöz, A. S. (2017). Integrating meaning into quality evaluation of machine translation. 15TH CONFERENCE OF THE EUROPEAN CHAPTER OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS (EACL 2017), VOL 1: LONG PAPERS, 210–219. Association for Computational Linguistics (ACL).
- Chicago author-date
- Başkaya, Osman, Eray Yildiz, Doruk Tunaoğlu, Mustafa Tolga Eren, and A. Seza Doğruöz. 2017. “Integrating Meaning into Quality Evaluation of Machine Translation.” In 15TH CONFERENCE OF THE EUROPEAN CHAPTER OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS (EACL 2017), VOL 1: LONG PAPERS, 210–19. Association for Computational Linguistics (ACL).
- Chicago author-date (all authors)
- Başkaya, Osman, Eray Yildiz, Doruk Tunaoğlu, Mustafa Tolga Eren, and A. Seza Doğruöz. 2017. “Integrating Meaning into Quality Evaluation of Machine Translation.” In 15TH CONFERENCE OF THE EUROPEAN CHAPTER OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS (EACL 2017), VOL 1: LONG PAPERS, 210–219. Association for Computational Linguistics (ACL).
- Vancouver
- 1.Başkaya O, Yildiz E, Tunaoğlu D, Eren MT, Doğruöz AS. Integrating meaning into quality evaluation of machine translation. In: 15TH CONFERENCE OF THE EUROPEAN CHAPTER OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS (EACL 2017), VOL 1: LONG PAPERS. Association for Computational Linguistics (ACL); 2017. p. 210–9.
- IEEE
- [1]O. Başkaya, E. Yildiz, D. Tunaoğlu, M. T. Eren, and A. S. Doğruöz, “Integrating meaning into quality evaluation of machine translation,” in 15TH CONFERENCE OF THE EUROPEAN CHAPTER OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS (EACL 2017), VOL 1: LONG PAPERS, Valencia, Spain, 2017, pp. 210–219.
@inproceedings{8694738,
abstract = {{Machine translation (MT) quality is evaluated through comparisons between MT outputs and the human translations (HT). Traditionally, this evaluation relies on form related features (e.g. lexicon and syntax) and ignores the transfer of meaning reflected in HT outputs. Instead, we evaluate the quality of MT outputs through meaning related features (e.g. polarity, subjectivity) with two experiments. In the first experiment, the meaning related features are compared to human rankings individually. In the second experiment, combinations of meaning related features and other quality metrics are utilized to predict the same human rankings. The results of our experiments confirm the benefit of these features in predicting human evaluation of translation quality in addition to traditional metrics which focus mainly on form.}},
author = {{Başkaya, Osman and Yildiz, Eray and Tunaoğlu, Doruk and Eren, Mustafa Tolga and Doğruöz, A. Seza}},
booktitle = {{15TH CONFERENCE OF THE EUROPEAN CHAPTER OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS (EACL 2017), VOL 1: LONG PAPERS}},
isbn = {{9781945626340}},
keywords = {{Machine Translation quality,form and meaning in Machine Translation,MT quality metrics,lt3}},
language = {{eng}},
location = {{Valencia, Spain}},
pages = {{210--219}},
publisher = {{Association for Computational Linguistics (ACL)}},
title = {{Integrating meaning into quality evaluation of machine translation}},
url = {{https://www.aclweb.org/anthology/E17-1020.pdf}},
year = {{2017}},
}