AI Powered Voice Agent Using NLP and ASR

Abstract

The rapid expansion of digital services has intensified the demand for intelligent, cost-effective, and scalable customer support systems. Traditional call centers are plagued by high operational costs, limited language support, and inconsistent service quality. This paper presents the design, implementation, and evaluation of an AI-Powered Voice Agent that leverages Automatic Speech Recognition (ASR), Natural Language Processing (NLP), and Large Language Models (LLMs) to automate call center operations in three Indian languages — Marathi, Hindi, and English. The system integrates the N8N workflow automation platform as a low-code orchestration engine enabling modular workflows for appointment scheduling, marketing campaigns, and query resolution. The architecture employs transformer-based ASR models, mBERT-based intent classification achieving 90%+ accuracy, and LLMs including ChatGPT and Gemini for context-aware dialogue generation. Pilot deployment results demonstrate WER of 8–12%, response latency of 3.8 seconds, 92% customer satisfaction, and 65% operational cost reduction.