A Real-Time Sign Language Recognition System Using MediaPipe and Random Forest

Abstract

The communication gap between hearing and hearing-impaired individuals remains a persistent challenge in modern society, necessitating efficient and accessible assistive technologies. Existing sign language recognition systems often rely on computationally intensive deep learning models and static datasets, limiting their adaptability and real-time usability.

This paper presents a real-time, user-customizable Automated Sign Language Recognition System that combines efficient computer vision techniques with lightweight machine learning. The system utilizes MediaPipe for robust extraction of hand landmarks, generating normalized spatial features invariant to environmental variations. These features are further processed using a Random Forest classifier, selected for its fast-training capability and effectiveness on non-linear data.

A key contribution of this work lies in its end-to-end pipeline that enables user-driven dataset creation, rapid model training, and real-time gesture recognition within a unified web-based interface. Unlike traditional approaches, the proposed system incorporates sequence-based feature aggregation from video inputs, enhancing robustness and stability in prediction.

Experimental observations indicate that the system achieves high accuracy (~90%) with low latency (<100 ms), making it suitable for real-time applications. Additionally, the integration of Text-to-Speech functionality improves accessibility by enabling seamless communication. The proposed approach offers a scalable and efficient alternative to deep learning-based systems, particularly for personalized and resource-constrained environments.