AI-Based Detection of Cloned Voices in Deepfake Videos

Abstract

Voice cloning is no longer science fiction. AI tools today can copy someone's voice from just a few seconds of audio. This creates a new threat: take a real video of a trusted person, swap in a cloned voice saying something false, and share it. The face is real, the voice sounds real, but the message is fabricated. Existing speaker verification systems often miss it too. This paper describes a two-phase detection system built for exactly this kind of attack. Phase I analyzes audio using MFCC features, Mel-spectrograms, and a CNN-based classifier. Phase II adds video analysis, checking whether the speaker's lips match the audio and looking for timing mismatches. A custom dataset of real and cloned voice samples was built alongside benchmarks ASVspoof 2019/2021 and FakeAVCeleb. Results show cloned voices leave detectable traces, and combining both phases is noticeably more reliable than audio analysis alone.