Paper Title
CONFIDENCE-WEIGHTED MULTIMODAL AFFECT FUSION FOR EMOTIONALLY ADAPTIVE GENERATIVE RECOMMENDATION

Abstract
Digital recommendation engines deployed across media streaming, e-commerce, and adaptive learning platforms customarily derive user preferences from accumulated behavioural data, an approach that overlooks the transient affective states which materially influence moment-to-moment user decision-making. This paper introduces EmotionSense, a deployable end-to-end architecture that embeds real-time multimodal emotion recognition within a generative personalisation pipeline. The system extracts affective features concurrently from three input channels: natural language text via a fine-tuned transformer classifier, acoustic speech via a convolutional-recurrent neural network trained on log-mel spectrogram representations, and facial expressions via a convolutional neural network detecting five discrete emotion categories. A confidence-weighted decision-level fusion mechanism aggregates individual modality emotion probability distributions, dynamically prioritising channels with higher inferential reliability when environmental degradation reduces signal quality. The consolidated affect vector then conditions a large language model through structured emotion-aware prompting, yielding recommendations that align both in content and communicative tone with the detected emotional state. Evaluation across five representative interaction sessions demonstrates that fused confidence scores consistently surpass unimodal baselines, while generated outputs exhibit measurably superior emotional alignment, contextual appropriateness, and linguistic adaptability relative to emotion-agnostic recommendations. Keywords - Affective Computing, Confidence-Weighted Fusion, Generative Personalisation, Real-Time Emotion Recognition, Emotion-Conditioned Prompting.