LDW : Label Divergence Weighting for multimodal sentiment analysis
- Author
- Quanqi Du (UGent) , Loic De Langhe (UGent) , Els Lefever (UGent) and Veronique Hoste (UGent)
- Organization
- Project
- Abstract
- Multimodal sentiment analysis (MSA) traditionally assumes a unified emotional signal across modalities such as text, audio, and video. However, recent findings suggest that each modality may convey distinct affective perspectives. Motivated by perspectivist theories from cognitive science and natural language processing, this paper introduces Label Divergence Weighting (LDW), a modality-weighting strategy that dynamically adjusts trust in each modality based on its alignment with the overall sentiment label. The LDW framework leverages training-time supervision from the divergence between unimodal and multimodal sentiment annotations to learn modality reliability, and applies this learning to unseen data without requiring unimodal labels at inference time. Integrated into a multitask variant of the Tensor Fusion Network (MTFN), the proposed LDW-MTFN model achieves state-of-the-art results on both the acted Chinese dataset CH-SIMS and the authentic English dataset UniC. Extensive experiments and ablation studies demonstrate the robustness and generalizability of LDW across datasets with different cultural, linguistic, and environmental characteristics.
- Keywords
- Label divergence, Unimodal labels, Modality weighting, Multimodal sentiment analysis, Multimodal fusion
Downloads
-
(...).pdf
- full text (Published version)
- |
- UGent only
- |
- |
- 1.51 MB
Citation
Please use this url to cite or link to this publication: http://hdl.handle.net/1854/LU-01K1KF7AE2824FY7PC3ZHKV0ZV
- MLA
- Du, Quanqi, et al. “LDW : Label Divergence Weighting for Multimodal Sentiment Analysis.” Proceedings of the 33rd ACM International Conference on Multimedia (MM 2025), Association for Computing Machinery (ACM), 2025, pp. 12342–51, doi:10.1145/3746027.3758160.
- APA
- Du, Q., De Langhe, L., Lefever, E., & Hoste, V. (2025). LDW : Label Divergence Weighting for multimodal sentiment analysis. Proceedings of the 33rd ACM International Conference on Multimedia (MM 2025), 12342–12351. https://doi.org/10.1145/3746027.3758160
- Chicago author-date
- Du, Quanqi, Loic De Langhe, Els Lefever, and Veronique Hoste. 2025. “LDW : Label Divergence Weighting for Multimodal Sentiment Analysis.” In Proceedings of the 33rd ACM International Conference on Multimedia (MM 2025), 12342–51. New York: Association for Computing Machinery (ACM). https://doi.org/10.1145/3746027.3758160.
- Chicago author-date (all authors)
- Du, Quanqi, Loic De Langhe, Els Lefever, and Veronique Hoste. 2025. “LDW : Label Divergence Weighting for Multimodal Sentiment Analysis.” In Proceedings of the 33rd ACM International Conference on Multimedia (MM 2025), 12342–12351. New York: Association for Computing Machinery (ACM). doi:10.1145/3746027.3758160.
- Vancouver
- 1.Du Q, De Langhe L, Lefever E, Hoste V. LDW : Label Divergence Weighting for multimodal sentiment analysis. In: Proceedings of the 33rd ACM International Conference on Multimedia (MM 2025). New York: Association for Computing Machinery (ACM); 2025. p. 12342–51.
- IEEE
- [1]Q. Du, L. De Langhe, E. Lefever, and V. Hoste, “LDW : Label Divergence Weighting for multimodal sentiment analysis,” in Proceedings of the 33rd ACM International Conference on Multimedia (MM 2025), Dublin, Ireland, 2025, pp. 12342–12351.
@inproceedings{01K1KF7AE2824FY7PC3ZHKV0ZV,
abstract = {{Multimodal sentiment analysis (MSA) traditionally assumes a unified emotional signal across modalities such as text, audio, and video. However, recent findings suggest that each modality may convey distinct affective perspectives. Motivated by perspectivist theories from cognitive science and natural language processing, this paper introduces Label Divergence Weighting (LDW), a modality-weighting strategy that dynamically adjusts trust in each modality based on its alignment with the overall sentiment label. The LDW framework leverages training-time supervision from the divergence between unimodal and multimodal sentiment annotations to learn modality reliability, and applies this learning to unseen data without requiring unimodal labels at inference time. Integrated into a multitask variant of the Tensor Fusion Network (MTFN), the proposed LDW-MTFN model achieves state-of-the-art results on both the acted Chinese dataset CH-SIMS and the authentic English dataset UniC. Extensive experiments and ablation studies demonstrate the robustness and generalizability of LDW across datasets with different cultural, linguistic, and environmental characteristics.}},
author = {{Du, Quanqi and De Langhe, Loic and Lefever, Els and Hoste, Veronique}},
booktitle = {{Proceedings of the 33rd ACM International Conference on Multimedia (MM 2025)}},
isbn = {{9798400720352}},
keywords = {{Label divergence,Unimodal labels,Modality weighting,Multimodal sentiment analysis,Multimodal fusion}},
language = {{eng}},
location = {{Dublin, Ireland}},
pages = {{12342--12351}},
publisher = {{Association for Computing Machinery (ACM)}},
title = {{LDW : Label Divergence Weighting for multimodal sentiment analysis}},
url = {{http://doi.org/10.1145/3746027.3758160}},
year = {{2025}},
}
- Altmetric
- View in Altmetric
- Web of Science
- Times cited: