Situation awareness in ai-based technologies and multimodal systems: Architectures, challenges and applications

J Chen, KP Seng, J Smith, LM Ang - IEEE Access, 2024 - ieeexplore.ieee.org
Situation Awareness (SA) is a process of sensing, understanding and predicting the
environment and is an important component in complex systems. The reception of …

Memory-aware continual learning with multi-modal social media streams for unsupervised disaster classification

Y Mao, X Yan, Z Hu, X Zhang, Y Ye, H Yu - Advanced Engineering …, 2024 - Elsevier
Social media has emerged as a major hub for information dissemination during disasters. It
serves as a central repository for massive real-time updates about disasters, facilitating the …

Multimodal semi-supervised learning for disaster tweet classification

I Sirbu, T Sosea, C Caragea, D Caragea… - Proceedings of the …, 2022 - aclanthology.org
During natural disasters, people often use social media platforms, such as Twitter, to post
information about casualties and damage produced by disasters. This information can help …

Understanding image-text relations and news values for multimodal news analysis

GS Cheema, S Hakimov, E Müller-Budack… - Frontiers in artificial …, 2023 - frontiersin.org
The analysis of news dissemination is of utmost importance since the credibility of
information and the identification of disinformation and misinformation affect society as a …

Crisisvit: A robust vision transformer for crisis image classification

Z Long, R Mccreadie, M Imran - arXiv preprint arXiv:2401.02838, 2024 - arxiv.org
In times of emergency, crisis response agencies need to quickly and accurately assess the
situation on the ground in order to deploy relevant services and resources. However …

Infrastructure ombudsman: Mining future failure concerns from structural disaster response

MTA Chowdhury, S Datta, N Sharma… - Proceedings of the …, 2024 - dl.acm.org
Current research concentrates on studying discussions on social media related to structural
failures to improve disaster response strategies. However, detecting social web posts …

Images Connect Us Together: Navigating a COVID-19 Local Outbreak in China Through Social Media Images

C He, L He, W Yang, B Li - Proceedings of the ACM on Human-Computer …, 2024 - dl.acm.org
Social media images, curated or casual, have become a crucial component of
communicating situational information and emotions during health crises. Despite its …

C-CLIP: Contrastive Image-Text Encoders to Close the Descriptive-Commentative Gap

W Theisen, WJ Scheirer - Proceedings of the IEEE/CVF …, 2024 - openaccess.thecvf.com
The interplay between the image and comment on a social media post is one of high
importance for understanding its overall message. Recent strides in multimodal embedding …

Contrastive Learning for Multimodal Classification of Crisis related Tweets

B Mandal, S Khanal, D Caragea - Proceedings of the ACM on Web …, 2024 - dl.acm.org
Multimodal tasks require learning a joint representation of the constituent modalities of data.
Contrastive learning learns a joint representation by using a contrastive loss. For example …

Unsupervised multimodal learning for image-text relation classification in tweets

L Sun, Q Li, L Liu, Y Su - Pattern Analysis and Applications, 2023 - Springer
Recent studies show that the use of multimodality can effectively enhance the understanding
of social media content. The relations between texts and images become an important basis …