MariATE : automatic term extraction using large language models in the maritime domain
- Author
- Shijie Liu, Els Lefever (UGent) and Veronique Hoste (UGent)
- Organization
- Abstract
- This study presents a comprehensive evaluation of Large Language Models (LLMs) for automatic term extraction in the maritime safety domain. The research examines the zero-shot performance of seven state-of-the-art LLMs, including both open-source and closed-source models, and investigates terminology annotation strategies for optimal coverage. Nested annotation captures both complete technical expressions and their constituent components, while full-term annotation focuses exclusively on maximal-length terms. Experimental results demonstrate Claude-3.5-Sonnet's superior performance (F1-score of 0.80) in maritime safety terminology extraction, particularly in boundary detection capabilities. Error analysis reveals three primary challenges: distinguishing contextual descriptions from legitimate terminology, handling complex multi-word expressions, and identifying maritime safety operational and navigational terms. Analysis of annotation strategies reveals that the full-term annotation approach achieves 95.24% coverage of unique terms compared to the nested annotation approach. The additional 4.76% of terms identified through nested annotation represents subcomponents of larger technical expressions. These findings advance the understanding of LLMs' capabilities in specialized terminology extraction and provide empirical evidence supporting the sufficiency of full-term annotation for comprehensive terminology coverage in domain-specific applications.
- Keywords
- computational linguistics, computational terminology, automatic term extraction, information retrieval, large language models, zero-shot
Downloads
-
publisher version.pdf
- full text (Published version)
- |
- open access
- |
- |
- 538.35 KB
Citation
Please use this url to cite or link to this publication: http://hdl.handle.net/1854/LU-01K09EPY8D9TC98KN10JJ7742G
- MLA
- Liu, Shijie, et al. “MariATE : Automatic Term Extraction Using Large Language Models in the Maritime Domain.” Proceedings of the 15th International Conference on Recent Advances in Natural Language Processing, edited by Galia Angelova et al., INCOMA Ltd., 2025, pp. 663–73, doi:10.26615/978-954-452-098-4-077.
- APA
- Liu, S., Lefever, E., & Hoste, V. (2025). MariATE : automatic term extraction using large language models in the maritime domain. In G. Angelova, M. Kunilovskaya, M. Escribe, & R. Mitkov (Eds.), Proceedings of the 15th International Conference on Recent Advances in Natural Language Processing (pp. 663–673). https://doi.org/10.26615/978-954-452-098-4-077
- Chicago author-date
- Liu, Shijie, Els Lefever, and Veronique Hoste. 2025. “MariATE : Automatic Term Extraction Using Large Language Models in the Maritime Domain.” In Proceedings of the 15th International Conference on Recent Advances in Natural Language Processing, edited by Galia Angelova, Maria Kunilovskaya, Marie Escribe, and Ruslan Mitkov, 663–73. Shoumen, Bulgaria: INCOMA Ltd. https://doi.org/10.26615/978-954-452-098-4-077.
- Chicago author-date (all authors)
- Liu, Shijie, Els Lefever, and Veronique Hoste. 2025. “MariATE : Automatic Term Extraction Using Large Language Models in the Maritime Domain.” In Proceedings of the 15th International Conference on Recent Advances in Natural Language Processing, ed by. Galia Angelova, Maria Kunilovskaya, Marie Escribe, and Ruslan Mitkov, 663–673. Shoumen, Bulgaria: INCOMA Ltd. doi:10.26615/978-954-452-098-4-077.
- Vancouver
- 1.Liu S, Lefever E, Hoste V. MariATE : automatic term extraction using large language models in the maritime domain. In: Angelova G, Kunilovskaya M, Escribe M, Mitkov R, editors. Proceedings of the 15th International Conference on Recent Advances in Natural Language Processing. Shoumen, Bulgaria: INCOMA Ltd.; 2025. p. 663–73.
- IEEE
- [1]S. Liu, E. Lefever, and V. Hoste, “MariATE : automatic term extraction using large language models in the maritime domain,” in Proceedings of the 15th International Conference on Recent Advances in Natural Language Processing, Varna, Bulgaria, 2025, pp. 663–673.
@inproceedings{01K09EPY8D9TC98KN10JJ7742G,
abstract = {{This study presents a comprehensive evaluation of Large Language Models (LLMs) for automatic term extraction in the maritime safety domain. The research examines the zero-shot performance of seven state-of-the-art LLMs, including both open-source and closed-source models, and investigates terminology annotation strategies for optimal coverage. Nested annotation captures both complete technical expressions and their constituent components, while full-term annotation focuses exclusively on maximal-length terms. Experimental results demonstrate Claude-3.5-Sonnet's superior performance (F1-score of 0.80) in maritime safety terminology extraction, particularly in boundary detection capabilities. Error analysis reveals three primary challenges: distinguishing contextual descriptions from legitimate terminology, handling complex multi-word expressions, and identifying maritime safety operational and navigational terms. Analysis of annotation strategies reveals that the full-term annotation approach achieves 95.24% coverage of unique terms compared to the nested annotation approach. The additional 4.76% of terms identified through nested annotation represents subcomponents of larger technical expressions. These findings advance the understanding of LLMs' capabilities in specialized terminology extraction and provide empirical evidence supporting the sufficiency of full-term annotation for comprehensive terminology coverage in domain-specific applications.}},
author = {{Liu, Shijie and Lefever, Els and Hoste, Veronique}},
booktitle = {{Proceedings of the 15th International Conference on Recent Advances in Natural Language Processing}},
editor = {{Angelova, Galia and Kunilovskaya, Maria and Escribe, Marie and Mitkov, Ruslan}},
isbn = {{9789544520984}},
issn = {{2603-2813}},
keywords = {{computational linguistics,computational terminology,automatic term extraction,information retrieval,large language models,zero-shot}},
language = {{eng}},
location = {{Varna, Bulgaria}},
pages = {{663--673}},
publisher = {{INCOMA Ltd.}},
title = {{MariATE : automatic term extraction using large language models in the maritime domain}},
url = {{http://doi.org/10.26615/978-954-452-098-4-077}},
year = {{2025}},
}
- Altmetric
- View in Altmetric