ASSAA: AI-Based Smart Surveillance System for Analysis and Alerting

Abstract

Conventional surveillance systems have been highly dependent on human intervention. They are thus prone to errors due to human factors such as fatigue, lack of concentration, and delayed reaction. ASSAA is an AI-driven intelligent surveillance system aimed at automating the real-time surveillance process. This is achieved through automated detection, recognition, and alerting modules. The ASSAA system analyzes real-time video streams by implementing a modular approach. It involves motion detection, person detection using YOLOv8, weapon detection, and facial recognition using dlib128-dimensional embedding vectors. Per- sons detected are checked against a watchlist that contains criminal history and citizenship information. The proposed solution provides a multi-level scoring model for threat assessment.  The algorithm incorporates the level of confidence in matching identities, presence of weapons, number of past convictions, and recency of recorded events. The output is a normalized threat score ranging from zero to one hundred. Threats are categorized into four levels, namely low, medium, high, and critical, depending on their risk scores. The system then generates alerts based on the severity of the detected threats. The performance of the ASSAA system has been val- idated experimentally using a simulated webcam environment. The current version of the ASSAA system exists as a command-line prototype.