UniC : a dataset for emotion analysis of videos with multimodal and unimodal labels
- Author
- Quanqi Du (UGent) , Sofie Labat (UGent) , Thomas Demeester (UGent) and Veronique Hoste (UGent)
- Organization
- Project
- Abstract
- Emotion is a key characteristic that differentiates humans from machines. It is intricate, encompassing a wide variety of emotional states, and is expressed through both verbal and non-verbal communication channels. Different modalities contribute in unique ways to the integrated expression of emotion. However, in most of the existing multimodal datasets, there is only one unified emotion label for the various modalities, ignoring the heterogeneity and complementarity of the different modalities. To bridge this gap, we introduce UniC, a novel multimodal emotion dataset featuring both integrated multimodal labels and independent unimodal labels. UniC is comprised of 965 emotion-rich video clips selected from YouTube, annotated in text, audio, silent video, and multimodal setups with both categorical and dimensional labels. We outline the steps taken to construct the dataset and analyze different modality perspectives in UniC. Our findings indicate that while in most cases the modality of text shares more emotional resemblance with the multimodal setup, other modalities can exhibit different, sometimes even opposite emotions that might contribute more to the overall emotion state. This dataset offers a modality-specific perspective on multimodal emotion analysis and has the potential to provide valuable insights for further research in human emotion understanding.
- Keywords
- Unimodal and multimodal labels, Text, Audio, Video, Sentiment and emotion modelling, Speech, RECOGNITION
Downloads
-
(...).pdf
- full text (Accepted manuscript)
- |
- UGent only (changes to open access on 2026-05-21)
- |
- |
- 14.04 MB
-
(...).pdf
- full text (Published version)
- |
- UGent only
- |
- |
- 3.66 MB
Citation
Please use this url to cite or link to this publication: http://hdl.handle.net/1854/LU-01JVSYA32J8EVCFDR66041QTG9
- MLA
- Du, Quanqi, et al. “UniC : A Dataset for Emotion Analysis of Videos with Multimodal and Unimodal Labels.” LANGUAGE RESOURCES AND EVALUATION, vol. 59, no. 3, 2025, pp. 2857–92, doi:10.1007/s10579-025-09837-0.
- APA
- Du, Q., Labat, S., Demeester, T., & Hoste, V. (2025). UniC : a dataset for emotion analysis of videos with multimodal and unimodal labels. LANGUAGE RESOURCES AND EVALUATION, 59(3), 2857–2892. https://doi.org/10.1007/s10579-025-09837-0
- Chicago author-date
- Du, Quanqi, Sofie Labat, Thomas Demeester, and Veronique Hoste. 2025. “UniC : A Dataset for Emotion Analysis of Videos with Multimodal and Unimodal Labels.” LANGUAGE RESOURCES AND EVALUATION 59 (3): 2857–92. https://doi.org/10.1007/s10579-025-09837-0.
- Chicago author-date (all authors)
- Du, Quanqi, Sofie Labat, Thomas Demeester, and Veronique Hoste. 2025. “UniC : A Dataset for Emotion Analysis of Videos with Multimodal and Unimodal Labels.” LANGUAGE RESOURCES AND EVALUATION 59 (3): 2857–2892. doi:10.1007/s10579-025-09837-0.
- Vancouver
- 1.Du Q, Labat S, Demeester T, Hoste V. UniC : a dataset for emotion analysis of videos with multimodal and unimodal labels. LANGUAGE RESOURCES AND EVALUATION. 2025;59(3):2857–92.
- IEEE
- [1]Q. Du, S. Labat, T. Demeester, and V. Hoste, “UniC : a dataset for emotion analysis of videos with multimodal and unimodal labels,” LANGUAGE RESOURCES AND EVALUATION, vol. 59, no. 3, pp. 2857–2892, 2025.
@article{01JVSYA32J8EVCFDR66041QTG9,
abstract = {{Emotion is a key characteristic that differentiates humans from machines. It is intricate, encompassing a wide variety of emotional states, and is expressed through both verbal and non-verbal communication channels. Different modalities contribute in unique ways to the integrated expression of emotion. However, in most of the existing multimodal datasets, there is only one unified emotion label for the various modalities, ignoring the heterogeneity and complementarity of the different modalities. To bridge this gap, we introduce UniC, a novel multimodal emotion dataset featuring both integrated multimodal labels and independent unimodal labels. UniC
is comprised of 965 emotion-rich video clips selected from YouTube, annotated in text, audio, silent video, and multimodal setups with both categorical and dimensional labels. We outline the steps taken to construct the dataset and analyze different modality perspectives in UniC. Our findings indicate that while in most cases the modality of text shares more emotional resemblance with the multimodal setup, other modalities can exhibit different, sometimes even opposite emotions that might contribute more to the overall emotion state. This dataset offers a modality-specific perspective on multimodal emotion analysis and has the potential to provide valuable insights for further research in human emotion understanding.}},
author = {{Du, Quanqi and Labat, Sofie and Demeester, Thomas and Hoste, Veronique}},
issn = {{1574-020X}},
journal = {{LANGUAGE RESOURCES AND EVALUATION}},
keywords = {{Unimodal and multimodal labels,Text,Audio,Video,Sentiment and emotion modelling,Speech,RECOGNITION}},
language = {{eng}},
number = {{3}},
pages = {{2857--2892}},
title = {{UniC : a dataset for emotion analysis of videos with multimodal and unimodal labels}},
url = {{http://doi.org/10.1007/s10579-025-09837-0}},
volume = {{59}},
year = {{2025}},
}
- Altmetric
- View in Altmetric
- Web of Science
- Times cited: