Automated Software Vulnerability Detection using Large Language Models: A Comparative Study from Cloud to Edge

Abstract

Recent developments in software systems have escalated the quantity of security vulnerabilities. Traditional Static Application Security Testing (SAST) tools rely only on method matching which can lead to a high number of false positives, as they are limited and cannot assess complex and context dependent vulnerabilities. With this limitation there was a need to investigate more advanced techniques for performing semantic code analysis. This study provides a detailed comparison of Large Language Models (LLMs) for automatically detecting vulnerabilities for both cloud and edge deployment environments. A multistage experimental pipeline was developed to evaluate the accuracy of a privacy preserving edge deployment using the Meta Llama 3 (8B) model running on consumer grade hardware (NVIDIA RTX 3060) against using Google Gemini 2.5 Flash for cloud-based inference. The methodology was developed using a hybrid dataset combining 2,000 annotated samples from both Microsoft’s CodeXGLUE benchmark and publicly available web vulnerability sources for standard vulnerability categories including SQL Injection, XSS, Buffer Overflows and Memory Leaks. Three prompting strategies (Zero-Shot, Role Based and Few-Shot prompting) were evaluated in this study. The results of the experiments show that the Few-Shot model trained locally achieved a recall of 96.0%, precision of 76.5% and an F1-score of 85.2%. In addition, this optimized local model outperformed the cloud baseline in overall accuracy (83.56% vs. 81.0%) while still preserving data privacy on-premise.