AI-Based Real-Time Multi-Object Detection and Adaptive Audio Generation

Abstract

This research presents AI Vision Application a fully local, browser-based system for real-time multi-object detection and adaptive English audio narration. The platform processes live camera feeds using YOLO-family models, implements lightweight object tracking for temporal continuity, applies user-configurable filters and confidence thresholds, renders annotated scenes in an interactive web dashboard, and generates concise, non-repetitive spoken summaries via browser speech synthesis. Designed for privacy and offline operation, it supports assistive technologies, educational demonstrations, laboratory monitoring, and general situational awareness without relying on cloud services. The methodology integrates computer vision pipelines, real-time Socket.IO communication, adaptive audio policies with stability and repetition control, session persistence, searchable exports, and formal benchmarking. All functional and integration tests passed successfully. Performance evaluation leverages official Ultralytics YOLO26 benchmarks, where the nano variant delivers 40.9 map (50-95) at ~25 FPS on CPU (38.9 ms latency, 2.4M parameters), scaling up to 57.5 map in larger variants. Results confirm that meaningful user experience arises from the orchestration of tracking, selective narration, and persistent analytics rather than detection accuracy alone. The system fills key gaps in assistive computer vision while remaining production ready. Limitations and future extensions, including enhanced tracking and multilingual support, are discussed.