Paper Title
A Review on Various Approaches to Multimodal Crisis Classification from Social Media Posts
Abstract
During crises, social media serves as a crucial channel for real-time updates, empowering humanitarian organizations to enhance their response speed and deliver the necessary resources efficiently. The focus is on extracting data from diverse modalities, including text, images, and videosfrom social media tweets and posts to identify whether they are related to any of the crisis related events, classify them and finally evaluate the extent of damage caused by the event. A comparative study on various image feature extraction models and text feature extraction models are discussed and also various multimodal fusion techniques used in capturing the interdependencies between these modalities are also studied. Building on these insights, we propose a model using DeBERTa for text feature extraction and ResNet50 for image features, fused through a cross-attention mechanism. This combined representation enhances classification performance.
Keywords - Crisis, Social Media, Disaster Response, DeBERT a, ResNet50, Cross Attention, Complementary Information, Multimodal Fusion, Multimodal models, TextFeature Extraction, Image feature extraction.