AI-Based Emotion Recognition and Supportive Response System for Non-Verbal Communication

Abstract

This research presents an AI-based multimodal therapist system for non-verbal communication using real-time facial emotion recognition and hand gesture detection. The system combines computer vision and deep learning techniques, where emotions are detected using a CNN model and gestures are recognized using MediaPipe and LSTM. Live video input is captured through a webcam and processed using OpenCV. The results are displayed via a Streamlit interface with voice feedback. Unlike existing systems, this approach integrates both emotion and gesture recognition, improving interaction accuracy and usability in real-time environments.