GENDEROUS : machine translation and cross-linguistic evaluation of a gender-ambiguous dataset
- Author
- Janica Hackenbuchner (UGent) , Eleni Gkovedarou (UGent) and Joke Daems (UGent)
- Organization
- Project
- Abstract
- Contributing to research on gender beyond the binary, this work introduces GENDEROUS, a dataset of gender-ambiguous sentences containing gender-marked occupations and adjectives, and sentences with the ambiguous or non-binary pronoun their. We cross-linguistically evaluate how machine translation (MT) systems and large language models (LLMs) translate these sentences from English into four grammatical gender languages: Greek, German, Spanish and Dutch. We show the systems’ continued default to male-gendered translations, with exceptions (particularly for Dutch). Prompting for alternatives, however, shows potential in attaining more diverse and neutral translations across all languages. An LLM-as-a-judge approach was implemented, where benchmarking against gold standards emphasises the continued need for human annotations.
Downloads
-
(...).pdf
- full text
- |
- UGent only
- |
- |
- 467.42 KB
Citation
Please use this url to cite or link to this publication: http://hdl.handle.net/1854/LU-01K94X05N04FBZC4FJN21T6FBT
- MLA
- Hackenbuchner, Janica, et al. “GENDEROUS : Machine Translation and Cross-Linguistic Evaluation of a Gender-Ambiguous Dataset.” NITS Conference 2025, Abstracts, 2025.
- APA
- Hackenbuchner, J., Gkovedarou, E., & Daems, J. (2025). GENDEROUS : machine translation and cross-linguistic evaluation of a gender-ambiguous dataset. NITS Conference 2025, Abstracts. Presented at the Network of Interdisciplinary Translation Studies in the Netherlands and Flanders (NITS), Tilburg, The Netherlands.
- Chicago author-date
- Hackenbuchner, Janica, Eleni Gkovedarou, and Joke Daems. 2025. “GENDEROUS : Machine Translation and Cross-Linguistic Evaluation of a Gender-Ambiguous Dataset.” In NITS Conference 2025, Abstracts.
- Chicago author-date (all authors)
- Hackenbuchner, Janica, Eleni Gkovedarou, and Joke Daems. 2025. “GENDEROUS : Machine Translation and Cross-Linguistic Evaluation of a Gender-Ambiguous Dataset.” In NITS Conference 2025, Abstracts.
- Vancouver
- 1.Hackenbuchner J, Gkovedarou E, Daems J. GENDEROUS : machine translation and cross-linguistic evaluation of a gender-ambiguous dataset. In: NITS conference 2025, Abstracts. 2025.
- IEEE
- [1]J. Hackenbuchner, E. Gkovedarou, and J. Daems, “GENDEROUS : machine translation and cross-linguistic evaluation of a gender-ambiguous dataset,” in NITS conference 2025, Abstracts, Tilburg, The Netherlands, 2025.
@inproceedings{01K94X05N04FBZC4FJN21T6FBT,
abstract = {{Contributing to research on gender beyond the binary, this work introduces GENDEROUS, a dataset of gender-ambiguous sentences containing gender-marked occupations and adjectives, and sentences with the ambiguous or non-binary pronoun their. We cross-linguistically evaluate how machine translation (MT) systems and large language models (LLMs) translate these sentences from English into four grammatical gender languages: Greek, German, Spanish and Dutch. We show the systems’ continued default to male-gendered translations, with exceptions (particularly for Dutch). Prompting for alternatives, however, shows potential in attaining more diverse and neutral translations across all languages. An LLM-as-a-judge approach was implemented, where benchmarking against gold standards emphasises the continued need for human annotations.}},
author = {{Hackenbuchner, Janica and Gkovedarou, Eleni and Daems, Joke}},
booktitle = {{NITS conference 2025, Abstracts}},
language = {{eng}},
location = {{Tilburg, The Netherlands}},
title = {{GENDEROUS : machine translation and cross-linguistic evaluation of a gender-ambiguous dataset}},
url = {{https://nitsnetwork.github.io/programme.html}},
year = {{2025}},
}