Multi-Domain Chatbot Development for Customer Service Using Pretrained Transformers and Multimodal Data

Abstract

Customer service automation demands systems capable of handling user queries in multiple formats beyond plain text. This work presents PyChat, a multi-domain, multimodal chatbot designed to process text, image, and audio inputs using pretrained transformer models. The system integrates DistilGPT-2 for natural language generation, BLIP VQA for visual question answering, and a speech recognition module for voice-to-text conversion. To reduce memory usage, 4-bit quantization using the BitsAndBytes library is applied without degrading model performance. The chatbot is deployed with Streamlit for real-time interaction. Experimental evaluation across 60 mixed-modality queries showed that quantization reduced GPU memory usage by 65% and maintained response accuracy above 87%. The results indicate that PyChat can efficiently perform domain-aware query handling and improve user engagement under constrained computational resources.