Multimodal classification: Current landscape, taxonomy and future directions

WC Sleeman IV, R Kapoor, P Ghosh - ACM Computing Surveys, 2022 - dl.acm.org
Multimodal classification research has been gaining popularity with new datasets in
domains such as satellite imagery, biometrics, and medicine. Prior research has shown the …

A review on methods and applications in multimodal deep learning

S Jabeen, X Li, MS Amin, O Bourahla, S Li… - ACM Transactions on …, 2023 - dl.acm.org
Deep Learning has implemented a wide range of applications and has become increasingly
popular in recent years. The goal of multimodal deep learning (MMDL) is to create models …

Recent advances and trends in multimodal deep learning: A review

J Summaira, X Li, AM Shoib, S Li, J Abdul - arXiv preprint arXiv …, 2021 - arxiv.org
Deep Learning has implemented a wide range of applications and has become increasingly
popular in recent years. The goal of multimodal deep learning is to create models that can …

Privacy enhanced multimodal neural representations for emotion recognition

M Jaiswal, EM Provost - Proceedings of the AAAI Conference on …, 2020 - ojs.aaai.org
Many mobile applications and virtual conversational agents now aim to recognize and adapt
to emotions. To enable this, data are transmitted from users' devices and stored on central …

Speaker-invariant adversarial domain adaptation for emotion recognition

Y Yin, B Huang, Y Wu, M Soleymani - Proceedings of the 2020 …, 2020 - dl.acm.org
Automatic emotion recognition methods are sensitive to the variations across different
datasets and their performance drops when evaluated across corpora. We can apply …

Muse: a multimodal dataset of stressed emotion

M Jaiswal, CP Bara - Proceedings of the Twelfth Language Resources …, 2020 - par.nsf.gov
Endowing automated agents with the ability to provide support, entertainment and
interaction with human beings requires sensing of the users' affective state. These affective …

[PDF][PDF] Mind the gap: On the value of silence representations to lexical-based speech emotion recognition.

M Perez, M Jaiswal, M Niu, C Gorrostieta… - …, 2022 - researchgate.net
Speech timing and non-speech regions (here referred to as “silence”), often play a critical
role in the perception of spoken language. Silence represents an important paralinguistic …

Emotion Recognition in the Real-World: Passively Collecting and Estimating Emotions from Natural Speech Data of Individuals with Bipolar Disorder

EM Provost, SH Sperry, J Tavernor… - IEEE Transactions …, 2024 - ieeexplore.ieee.org
Emotions provide critical information regarding a person's health and well-being. Therefore,
the ability to track emotion and patterns in emotion over time could provide new …

X-Norm: Exchanging Normalization Parameters for Bimodal Fusion

Y Yin, J Xu, T Zu, M Soleymani - … of the 2022 International Conference on …, 2022 - dl.acm.org
Multimodal learning aims to process and relate information from different modalities to
enhance the model's capacity for perception. Current multimodal fusion mechanisms either …

Fine-Grained Emotion Comprehension: Semisupervised Multimodal Emotion and Intensity Recognition

Z Fang, Z Liu, T Liu, CC Hung - IEEE Transactions on …, 2024 - ieeexplore.ieee.org
The rapid advancement of deep learning and the exponential growth of multimodal data
have led to increased attention on multimodal emotion analysis and comprehension in affect …