Word synchronization challenge : a benchmark for word association responses for large language models
(2025)
HUMAN-COMPUTER INTERACTION, HCI 2025, PT V.
In Lecture Notes in Computer Science
15770.
p.3-19
- Author
- Tanguy Cazalets (UGent) and Joni Dambre (UGent)
- Organization
- Abstract
- This paper introduces the Word Synchronization Challenge, a novel benchmark to evaluate large language models (LLMs) in Human-Computer Interaction (HCI). This benchmark utilizes a dynamic game-like framework to test LLMs’ ability to mimic human cognitive processes through word associations. By simulating complex human interactions, it assesses how LLMs interpret and align with human thought patterns during conversational exchanges, essential for effective social partnerships in HCI. Initial findings highlight the influence of model sophistication on performance, offering insights into the models’ capabilities to engage in meaningful social interactions and adapt behaviors in human-like manners. This research advances understanding of LLMs’ potential to replicate or diverge from human cognitive functions, paving the way for more nuanced and empathetic human-machine collaborations.
- Keywords
- Human Centered AI (HCAI), Explainable AI (XAI), Benchmark, Theory of Mind, Word associations
Downloads
-
(...).pdf
- full text (Accepted manuscript)
- |
- UGent only (changes to open access on 2026-06-05)
- |
- |
- 1.29 MB
-
(...).pdf
- full text (Published version)
- |
- UGent only
- |
- |
- 2.03 MB
Citation
Please use this url to cite or link to this publication: http://hdl.handle.net/1854/LU-01JVW243A2466QF51W74JW9DEH
- MLA
- Cazalets, Tanguy, and Joni Dambre. “Word Synchronization Challenge : A Benchmark for Word Association Responses for Large Language Models.” HUMAN-COMPUTER INTERACTION, HCI 2025, PT V, edited by Masaaki Kurosu and Ayako Hashizume, vol. 15770, Springer, 2025, pp. 3–19, doi:10.1007/978-3-031-93864-1_1.
- APA
- Cazalets, T., & Dambre, J. (2025). Word synchronization challenge : a benchmark for word association responses for large language models. In M. Kurosu & A. Hashizume (Eds.), HUMAN-COMPUTER INTERACTION, HCI 2025, PT V (Vol. 15770, pp. 3–19). https://doi.org/10.1007/978-3-031-93864-1_1
- Chicago author-date
- Cazalets, Tanguy, and Joni Dambre. 2025. “Word Synchronization Challenge : A Benchmark for Word Association Responses for Large Language Models.” In HUMAN-COMPUTER INTERACTION, HCI 2025, PT V, edited by Masaaki Kurosu and Ayako Hashizume, 15770:3–19. Springer. https://doi.org/10.1007/978-3-031-93864-1_1.
- Chicago author-date (all authors)
- Cazalets, Tanguy, and Joni Dambre. 2025. “Word Synchronization Challenge : A Benchmark for Word Association Responses for Large Language Models.” In HUMAN-COMPUTER INTERACTION, HCI 2025, PT V, ed by. Masaaki Kurosu and Ayako Hashizume, 15770:3–19. Springer. doi:10.1007/978-3-031-93864-1_1.
- Vancouver
- 1.Cazalets T, Dambre J. Word synchronization challenge : a benchmark for word association responses for large language models. In: Kurosu M, Hashizume A, editors. HUMAN-COMPUTER INTERACTION, HCI 2025, PT V. Springer; 2025. p. 3–19.
- IEEE
- [1]T. Cazalets and J. Dambre, “Word synchronization challenge : a benchmark for word association responses for large language models,” in HUMAN-COMPUTER INTERACTION, HCI 2025, PT V, Gothenburg, Sweden, 2025, vol. 15770, pp. 3–19.
@inproceedings{01JVW243A2466QF51W74JW9DEH,
abstract = {{This paper introduces the Word Synchronization Challenge, a novel benchmark to evaluate large language models (LLMs) in Human-Computer Interaction (HCI). This benchmark utilizes a dynamic game-like framework to test LLMs’ ability to mimic human cognitive processes through word associations. By simulating complex human interactions, it assesses how LLMs interpret and align with human thought patterns during conversational exchanges, essential for effective social partnerships in HCI. Initial findings highlight the influence of model sophistication on performance, offering insights into the models’ capabilities to engage in meaningful social interactions and adapt behaviors in human-like manners. This research advances understanding of LLMs’ potential to replicate or diverge from human cognitive functions, paving the way for more nuanced and empathetic human-machine collaborations.}},
author = {{Cazalets, Tanguy and Dambre, Joni}},
booktitle = {{HUMAN-COMPUTER INTERACTION, HCI 2025, PT V}},
editor = {{Kurosu, Masaaki and Hashizume, Ayako}},
isbn = {{9783031938634}},
issn = {{0302-9743}},
keywords = {{Human Centered AI (HCAI),Explainable AI (XAI),Benchmark,Theory of Mind,Word associations}},
language = {{eng}},
location = {{Gothenburg, Sweden}},
pages = {{3--19}},
publisher = {{Springer}},
title = {{Word synchronization challenge : a benchmark for word association responses for large language models}},
url = {{http://doi.org/10.1007/978-3-031-93864-1_1}},
volume = {{15770}},
year = {{2025}},
}
- Altmetric
- View in Altmetric
- Web of Science
- Times cited: