Securing Large Language Models Against Prompt Injection and Data Leakage Attacks

https://doi.org/10.5281/zenodo.19344116

Abstract

Large Language Models (LLMs) have rapidly become integral to modern artificial intelligence applications, including conversational agents, software development tools, intelligent tutoring systems, and enterprise decision-support platforms. Their ability to process and generate human-like text has enabled widespread adoption across multiple industries. However, the increasing deployment of LLMs in real-world environments has introduced significant security and privacy risks, particularly in the form of prompt injection attacks and sensitive data leakage. Prompt injection attacks occur when maliciously crafted inputs manipulate the model into overriding system instructions, bypassing safety policies, or revealing hidden prompts and confidential information. Similarly, data leakage vulnerabilities may expose proprietary knowledge, training data, or sensitive organizational information through model responses. To address these challenges, this study proposes a comprehensive security architecture designed to enhance the robustness of LLM systems against adversarial prompts and information leakage. The proposed framework integrates multiple defense layers, including prompt validation, contextual anomaly detection, adversarial training, input sanitization, and output filtering mechanisms. Additionally, reinforcement learning–based safety policies are incorporated to guide model responses and enforce security constraints during inference. Experimental evaluation was conducted using a dataset containing both benign queries and adversarial prompts designed to exploit LLM vulnerabilities. Results demonstrate that the proposed framework significantly improves model resilience compared to baseline LLM implementations. Specifically, the prompt injection attack success rate decreased from 62% to 12%, while the data leakage rate dropped from 55% to 9%. Furthermore, the system maintained high response accuracy and improved overall reliability under adversarial conditions. These findings highlight the importance of integrating security-aware design principles into LLM deployment pipelines to ensure trustworthy, secure, and responsible AI systems.