Speech-To-Text Summarization System For Smart Note-Taking

Abstract

The rapid convergence of Automatic Speech Recognition (ASR) and Natural Language Processing (NLP) has fundamentally transformed the digital accessibility landscape, enabling the development of sophisticated applications capable of transcribing and summarizing spoken content in real-time. This comprehensive research report investigates the architectural, linguistic, and computational frameworks essential for constructing a highly accurate Speech-to-Text and Text-to-Summarization application, with a specialized focus on multilingual and low-resource Indian languages such as Hindi and Marathi.¹ While traditional acoustic models have demonstrated remarkable proficiency in high-resource contexts, adapting these systems to linguistically diverse and morphologically complex environments presents profound technical challenges. Through a rigorous examination of state-of-the-art acoustic models—including OpenAI's Whisper, Wav2Vec 2.0, and Conformer—this study evaluates their feature encoding mechanisms, quantization processes, and adaptability to code-mixed speech.³ Concurrently, the report analyses advanced abstractive summarization engines like BART, T5, and PEGASUS, emphasizing their capacity to generate coherent, factually consistent summaries from conversational transcripts.¹ Special attention is devoted to the AI4Bharat ecosystem, specifically the Indic BART model, which leverages Devanagari script unification to overcome data scarcity in Indian language processing.⁸ The study further explores end-to-end system architecture, highlighting sophisticated evaluation metrics such as Word Error Rate (WER) and BERTScore.¹⁰ Finally, the report addresses the socio-technical implications of such applications, including their impact on educational cognitive load, user experience design paradigms, and the critical ethical considerations surrounding voice-inferred data privacy and algorithmic bias in automated transcription systems.¹²