Advanced search
Add to list

It's all in the eyes : using eye-tracking to assess the readability of machine translated literature

Toon Colman (UGent) , Margot Fonteyne (UGent) , Joke Daems (UGent) and Lieve Macken (UGent)
Author
Organization
Project
Abstract
With the arrival of neural machine translation (NMT) systems, translation quality has improved enormously. Despite these quality improvements, especially for more creative text types such as literary texts, remarkable differences can be observed when comparing human and machine translations. Webster et al. (2020) compared the modern Dutch human translations of four classic novels with their machine translated versions generated by Google Translate and DeepL and found that not only a large proportion of the machine translated sentences contained errors, but they also observed a lower level of lexical richness and local cohesion in the NMT output compared to the human translations. The most frequent errors observed in their data set were mistranslations (37%), coherence (32%), and style & register (13%) errors. This top three corresponds to previous research by Tezcan et al. (2019) and Fonteyne et al. (2020) who both discussed the quality of Agatha Christie’s The Mysterious Affair at Styles, translated by Google's neural machine translation system from English into Dutch. In this presentation we report on the first results of an eye-tracking study in which participants read the full novel (Agatha Christie's The Mysterious Affair at Styles), in Dutch alternating between a machine translation (MT) and a human translation (HT). We compare the reading process of participants reading both versions and analyse to what extent MT impacts the reading process. As a human annotator has marked and classified all errors in the machine translated version of the novel (Fonteyne et al. 2020) we also study which errors impact this reading process most. The data set expands the Ghent Eye-Tracking Corpus (Cop et al., 2017), which contains eye tracking data of participants reading Agatha Christie’s novel in English and in its Dutch (human) translation.
Keywords
LT3, literary machine translation, quality assessment, reading behaviour, eye-tracking, corpus study

Citation

Please use this url to cite or link to this publication:

MLA
Colman, Toon, et al. “It’s All in the Eyes : Using Eye-Tracking to Assess the Readability of Machine Translated Literature.” PETRA-E Conference : Literary Translation Studies Today and Tomorrow, 2021.
APA
Colman, T., Fonteyne, M., Daems, J., & Macken, L. (2021). It’s all in the eyes : using eye-tracking to assess the readability of machine translated literature. PETRA-E Conference : Literary Translation Studies Today and Tomorrow. Presented at the PETRA-E Conference : Literary Translation Studies Today and Tomorrow, Dublin, Ireland.
Chicago author-date
Colman, Toon, Margot Fonteyne, Joke Daems, and Lieve Macken. 2021. “It’s All in the Eyes : Using Eye-Tracking to Assess the Readability of Machine Translated Literature.” In PETRA-E Conference : Literary Translation Studies Today and Tomorrow.
Chicago author-date (all authors)
Colman, Toon, Margot Fonteyne, Joke Daems, and Lieve Macken. 2021. “It’s All in the Eyes : Using Eye-Tracking to Assess the Readability of Machine Translated Literature.” In PETRA-E Conference : Literary Translation Studies Today and Tomorrow.
Vancouver
1.
Colman T, Fonteyne M, Daems J, Macken L. It’s all in the eyes : using eye-tracking to assess the readability of machine translated literature. In: PETRA-E Conference : Literary Translation Studies Today and Tomorrow. 2021.
IEEE
[1]
T. Colman, M. Fonteyne, J. Daems, and L. Macken, “It’s all in the eyes : using eye-tracking to assess the readability of machine translated literature,” in PETRA-E Conference : Literary Translation Studies Today and Tomorrow, Dublin, Ireland, 2021.
@inproceedings{8738503,
  abstract     = {{With the arrival of neural machine translation (NMT) systems, translation quality has improved enormously. Despite these quality improvements, especially for more creative text types such as literary texts, remarkable differences can be observed when comparing human and machine translations. Webster et al. (2020) compared the modern Dutch human translations of four classic novels with their machine translated versions generated by Google Translate and DeepL and found that not only a large proportion of the machine translated sentences contained errors, but they also observed a lower level of lexical richness and local cohesion in the NMT output compared to the human translations. The most frequent errors observed in their data set were mistranslations (37%), coherence (32%), and style & register (13%) errors. This top three corresponds to previous research by Tezcan et al. (2019) and Fonteyne et al. (2020) who both discussed the quality of Agatha Christie’s The Mysterious Affair at Styles, translated by Google's neural machine translation system from English into Dutch. In this presentation we report on the first results of an eye-tracking study in which participants read the full novel (Agatha Christie's The Mysterious Affair at Styles), in Dutch alternating between a machine translation (MT) and a human translation (HT). We compare the reading process of participants reading both versions and analyse to what extent MT impacts the reading process. As a human annotator has marked and classified all errors in the machine translated version of the novel (Fonteyne et al. 2020) we also study which errors impact this reading process most. The data set expands the Ghent Eye-Tracking Corpus (Cop et al., 2017), which contains eye tracking data of participants reading Agatha Christie’s novel in English and in its Dutch (human) translation.}},
  author       = {{Colman, Toon and Fonteyne, Margot and Daems, Joke and Macken, Lieve}},
  booktitle    = {{PETRA-E Conference : Literary Translation Studies Today and Tomorrow}},
  keywords     = {{LT3,literary machine translation,quality assessment,reading behaviour,eye-tracking,corpus study}},
  language     = {{eng}},
  location     = {{Dublin, Ireland}},
  title        = {{It's all in the eyes : using eye-tracking to assess the readability of machine translated literature}},
  url          = {{https://www.youtube.com/watch?v=sTx6SM9tXpU}},
  year         = {{2021}},
}