AI Assisted Automated Software Testing and Vulnerability Detection Using Large Language Models

Abstract

The discipline of software quality assurance is undergoing a profound evolutionary transition as the rapid contraction of vulnerability weaponization timelines renders traditional manual testing and rigid programmatic scripting obsolete. This review article provides an exhaustive analysis of the paradigm shift brought by Large Language Models (LLMs) from automating repetitive testing actions to automating complex cognitive processes. We categorize foundational code-aware model architectures into encoder-only, encoder-decoder, and decoder-only archetypes, evaluating their trade-offs across execution latency, logical reasoning, and context window capacities. Furthermore, we analyze state-of-the-art automated test generation frameworks that integrate structural program analysis, Retrieval-Augmented Generation, and execution feedback loops, while highlighting the ongoing debate regarding whether LLMs possess true semantic reasoning capabilities or merely replicate memorized surface-level code patterns. In the realm of software security, this paper evaluates neuro-symbolic detection frameworks that combine the high recall of LLMs with the precision of static application security testing tools. We explore precision vulnerability localization techniques, including attention rectification mechanisms and agentic trace analysis, which resolve traditional granularity dilemmas by pinpointing line-level defects. Finally, we examine the evolution of automated program repair from brittle generate-and-validate templates into conversational, multi-turn feedback loops and autonomous multi-agent cyber-reasoning systems that utilize the Model Context Protocol and ReAct-style execution histories. Ultimately, this review establishes that while AI-driven testing introduces non-deterministic behaviors and security risks, its integration into autonomous lifecycles yields unprecedented efficiency, as demonstrated by the automated discovery and patching of real-world zero-day vulnerabilities in active critical infrastructure.