Multimodal detection and Severity Assessment of Autism Spectrum Disorder using ML and DL

Abstract

Autism Spectrum Disorder (ASD) is fundamentally challenging to diagnose early, as conventional methods rely heavily on subjective, time-intensive clinical assessments. This research addresses the urgent need for scalable, objective, and efficient approaches to enable timely detection and severity assessment, ensuring better intervention outcomes for children in the critical 18 to 72-month developmental window. The proposed framework bridges the gap between traditional observational checklists and automated computational diagnostics by utilizing a dual-layered multimodal system.

The methodology integrates a Machine Learning (ML) module for behavioral screening and a Deep Learning (DL) module for spatio-temporal video analysis. The ML component utilizes a Random Forest classifier to process clinical data, achieving a testing accuracy of 92%. Concurrently, the DL component employs a hybrid CNN-LSTM architecture to analyze Joint Attention (JA) behaviors in video sequences. By extracting spatial features via CNN and capturing temporal dependencies through LSTM, the system achieves a validation accuracy of ~96% and a testing accuracy of ~90% for binary classification (ASD vs. Non-ASD).

Results from this study indicate that the system is highly generalized and resistant to overfitting, even when operating on a constrained clinical dataset of 177 video samples. By integrating these models into a locally hosted "Early Steps" environment this project demonstrates a secure, privacy-compliant, and high-precision alternative to manual screening. This work provides a foundation for future real-time pediatric diagnostic tools that can be deployed in resource-limited settings.