owasp-vulnerable-llm-applications.md

title	date	tags	aliases
2024-04-29 - TIL - OWASP Vulnerable LLM Applications	2024-04-28 17:00:57 -0700	owasp-vulnerable-llm-applications resources til	today-i-learned things-i-learned

2024-04-29 - TIL - OWASP Vulnerable LLM Applications

Links to currently available vulnerable LLM applications that contain useful educational materials and training exercises relevant to the OWASP Top 10 for Large Language Model Applications project.

Vulnerable LLM Applications

• Lakera Gandalf - Test your prompting skills to make Gandalf reveal secret information (101 - Beginner Friendly)

• PortSwiger Web LLM Attacks - Deeper dive into LLM attacks (201 - Intermediate)

• @hxs220034/DamnVulnerableLLMApplication-Demo - Damn Vulnerable LLM Application (Requires OpenAI API Key)

• @hxs220034/SecureLLMCTF - Secure LLM CTF (Requires OpenAI API Key)

• @harishsg993010/SecureLLM - Secure LLM CTF Repo (Requires OpenAI API Key)

• HadessCS/Delta - LLM Vulnerable Application (Requires OpenAI API Key)

• wrongsecrets LLM challenges - Prompt AI to provide the secret uploaded to an LLM application (101 - Beginner Friendly)

  • Currently links to Lakera Gandalf
  • Previously linked to gpa.43z.one

• Rebuff Playground for prompt injection - Self-hardening prompt injection detector (Requires OpenAI API Key)

• @h43z GPT Prompt Attack - Test your GPT Prompt Attack skills (101 - Beginner Friendly)

• svenmorgenrothio Prompt Injection Playground - Prompt Injection Playground (1+ years since last update)

• Yudbot - Convince Yud to sell you a GPU cluster - Convince Yud to sell you a GPU cluster Prompt Attack (201 - Intermediate)

• AI Challenges with Prizes but requires lengthy login process and your own API key - AI Challenges with Prizes (301 - Advanced)

• Damn Vulnerable LLM Agent - CTF Lab Environment similar to Juice Shop but for experimenting with prompt injection attacks (301 - Advanced)

• Google CyberBotLLM Gemini - Similar to Damn Vulnerable LLM Agent and requires GOOGLE_APPLICATIONS_CREDENTIALS (301 - Advanced)

• Tensor Trust — open-source project for testing prompt injection - Hack other players' AI account and defend your own AI account (301 - Advanced but auth is currently broken)

Use Cases

• Example vulnerable LLM applications that contain useful educational materials and training exercises

References

• Vulnerable LLM Applications · OWASP/www-project-top-10-for-large-language-model-applications Wiki
• Educational Resources · OWASP/www-project-top-10-for-large-language-model-applications Wiki
• OWASP Top 10 for Large Language Model Applications
• OWASP | Top 10 for Large Language Model Applications