Prompt Shield: LLM Prompt Injection and Jailbreak Detector
Author
Gunaseelan S, Indra Kumar K S, Gowtham V, Harikrishna V, Mrs. Saranya M
Abstract
Large Language Models (LLMs) are now embedded in chatbots, coding assistants and automated workflows, where trusted developer instructions are combined with untrusted user text. Because both are written in ordinary language, a crafted prompt can override the intended rules, reveal hidden instructions or push the model toward unsafe behaviour, and conventional input validation does not detect such attacks. This paper presents Prompt Shield, an AI-powered gateway that inspects every prompt before it is forwarded to an LLM. The system is implemented in Python and Flask and combines a Term Frequency-Inverse Document Frequency (TF-IDF) and Logistic Regression classifier with a deterministic regular-expression engine of 31 attack signatures. When a signature matches a prompt that the classifier labelled Safe, the verdict is overridden to Dangerous and the confidence is raised to at least 85%. The classifier was trained on a labelled dataset of 1,361 prompts (697 Safe and 664 Dangerous) and achieved 98.53% accuracy on a stratified holdout partition of 273 samples. Every decision is recorded with its confidence, matched patterns and timestamp in a local SQLite database, with optional Supabase synchronization. Prompts classified as Dangerous are blocked before any external request is made, while Safe prompts are forwarded to the Google Gemini 2.5 Flash API. A Flask dashboard presents the verdict, confidence, detection details, live statistics and historical logs. The results show that a lightweight, transparent and auditable layer can provide a useful first line of defence for LLM applications, although its output should be treated as a defensive signal rather than a guarantee of detection.
Keywords
Prompt Injection, Jailbreak Detection, Large Language Models, TF-IDF, Logistic Regression, Regular Expressions, LLM Security
Full Text:
References
[1] OWASP Foundation. (n.d.). OWASP Top 10 for large language model applications: Prompt injection. https://owasp.org/www-project-top-10-for-large-language-model-applications/
[2] Pedregosa, F., et al. (2011). Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12, 2825–2830.
[3] Python Software Foundation. (n.d.). Python documentation. https://docs.python.org/
[4] Pallets Projects. (n.d.). Flask documentation. https://flask.palletsprojects.com/
[5] Google AI for Developers. (n.d.). Gemini API documentation. https://ai.google.dev/gemini-api/docs
[6] SQLite. (n.d.). SQLite documentation. https://www.sqlite.org/docs.html
[7] Supabase. (n.d.). Supabase documentation and REST API reference. https://supabase.com/docs
[8] Salton, G., & Buckley, C. (1988). Term-weighting approaches in automatic text retrieval. Information Processing & Management, 24(5), 513–523.
[9] Vaswani, A., et al. (2017). Attention is all you need. Advances in Neural Information Processing Systems, 30.
[10] Perez, F., & Ribeiro, I. (2022). Ignore previous prompt: Attack techniques for language models. arXiv:2211.09527.
[11] Greshake, K., et al. (2023). Not what you signed up for: Compromising real-world LLM-integrated applications with indirect prompt injection. arXiv:2302.12173.
[12] Wei, A., Haghtalab, N., & Steinhardt, J. (2023). Jailbroken: How does LLM safety training fail?. arXiv:2307.02483.
[13] Goodfellow, I., Shlens, J., & Szegedy, C. (2015). Explaining and harnessing adversarial examples. arXiv:1412.6572.
[14] Jurafsky, D., & Martin, J. H. (n.d.). Speech and language processing. https://web.stanford.edu/~jurafsky/slp3/