Project: M3-Apps: Enabling User-centric Modelling of Immersiveness in Multimodal Multimedia Applications

2021-11-01 – 2023-10-31

Abstract

Multimodal, immersive systems are the latest development within the field of multimedia. These systems emulate the senses by means of omnidirectional visuals, 360° sound, motion tracking and touch simulation to create a feeling of presence in the virtual environment. They have the potential to substitute physical interactions in application domains such as training (Industry 4.0), or e-health (tele-surgery). However, the current COVID19 pandemic has shown that they are not ready, as they do not fulfill the requirements in terms of immersiveness. In this research project, I argue that immersiveness can be devised from measuring four aspects, namely: presence (i.e. the feeling of being "in" the environment), cybersickness, network-related Quality of Experience and the usability of the application. Therefore, this project proposes a user-centric approach on the assessment and modelling of immersiveness. Two dimensions are put forward: (1) Subjective and objective assessment of presence, cybersickness, usability and QoE and (2) Real-time modelling of immersiveness. Furthermore, a proof-of-concept is designed including two use cases. As such, this work will significantly advance the state of the art on immersive systems and multimedia in general.