The OWASP AI Testing Guide: How to Operationalize It

Key takeaways

• The OWASP AI Testing Guide turns AI risk categories into 32 repeatable tests across applications, models, infrastructure and data. Use it alongside the OWASP Top 10 to check whether controls withstand the risks that taxonomy identifies.
• Launch approval is not lasting assurance, so repeat tests after material changes and throughout production. Shadow AI discovery and broader agent governance require coverage beyond the guide.
• Run the guide as a five-step lifecycle program: inventory and tier AI systems, name accountable owners, establish release gates, apply runtime controls and preserve audit evidence. Each step feeds the next, so test findings become enforceable policy rather than a one-time report.
• WitnessAI complements AI testing with network-level discovery, intent-based governance, runtime protection, and agentic security controls. For supported and integrated AI traffic, these capabilities help organizations apply policies during production use and retain evidence of policy decisions and enforcement across human and agent activity.
• Testing evidence supports compliance but does not establish EU AI Act compliance by itself. Map the guide to applicable obligations and retain metrics, test configurations and audit trails.

The OWASP AI Testing Guide gives enterprises an open, community-built catalog of 32 repeatable tests for AI systems, covering applications, models, infrastructure and data. Released as Version 1 in November 2025, it gives security, risk and AI teams one shared test vocabulary, giving security, risk, and AI teams a shared methodology for evaluating chatbots, copilots, and agents before deployment and as systems evolve.

The harder part is keeping those tests meaningful once systems are live. Customer-facing chatbots can make commitments a company is later held to, and AI agents call APIs and query databases on their own. A model that passed its tests at launch can behave differently after a provider update, a new tool connection or a prompt change.

This article explains what the guide contains, where a test-only approach leaves gaps, and a five-step model for running the guide as part of AI risk management. It also shows how runtime controls can complement test findings with ongoing policy enforcement across supported employee and agent workflows.

What is the OWASP AI Testing Guide?

The OWASP AI Testing Guide is a technology-agnostic methodology for evaluating AI system security and trustworthiness through structured testing. Test results provide evidence about assessed risks and controls, rather than a definitive determination that a system is trustworthy. Version 1, published on November 26, 2025, organizes 32 test cases into four layers. Matteo Meucci and Marco Morana lead the project, which OWASP labels an Incubator Project, and its stated audience includes developers, architects, auditors and risk officers.

Its scope reaches past classic security. The guide’s introduction argues that trustworthiness, not security alone, is the real objective for AI, and its named risk areas include bias, hallucinations, model drift and excessive agency alongside prompt injection and data poisoning.

The guide organizes 32 test cases across application, model, infrastructure, and data security. Before publication, verify the test counts, identifiers, and category descriptions against the specific OWASP version cited, and link directly to that version. Application tests cover prompt injection, sensitive data leaks, agentic behavior limits and hallucinations, while the model, infrastructure and data layers address evasion, poisoning, supply chain tampering, plugin boundary violations, runtime exfiltration, and data minimization and consent. According to the guide’s preface, each test links objectives, payloads and observable responses to remediation guidance, so results can be reported consistently across teams.

Why point-in-time testing leaves AI risk unmanaged

A successful test result reflects the system, configuration, and conditions evaluated at that point in time. Changes to models, prompts, integrations, and tools can alter the risk profile, making reassessment necessary. OWASP describes AI models as systems that fail in non-deterministic ways, and the prompt injection test, AITG-APP-01, warns that many techniques lose effectiveness over time or work only against certain models. The agentic behavior test is blunter, noting that prompt injection “can be mitigated but not completely prevented.”

Those failures carry business consequences. A British Columbia tribunal held Air Canada liable for its chatbot’s bad bereavement-fare advice, and agents raise the stakes further because they call APIs and query databases on their own, turning a missed weakness into an action rather than a bad answer.

Attack pressure is already measurable in a single Gartner survey.

• The survey, run from March to May 2025, covered 302 cybersecurity leaders. Of those, 32% reported an attack on AI applications through the application prompt, the exact surface the APP series tests.
• The same survey found that 69% of organizations suspect or have evidence that employees use prohibited public GenAI. That is Shadow AI, and the guide has no dedicated test for it.

Agentic AI and compliance also sit partly outside the guide. AITG-APP-06 is its main agent test, while separate OWASP projects carry broader agentic guidance. No official crosswalk maps the 32 tests to the EU AI Act, ISO/IEC 42001, DORA or SEC disclosure rules, so compliance teams build that mapping themselves.

Pre-deployment review, meanwhile, is becoming standard practice. The World Economic Forum’s Global Cybersecurity Outlook 2026 found that the share of organizations assessing the security of their AI tools rose from 37% in 2025 to 64% in 2026, with 40% running periodic reviews and 24% a one-time assessment. Periodic reviews can occur during production, but their effectiveness depends on frequency, scope, and whether material system changes trigger reassessment. Runtime monitoring and enforcement complement these reviews by addressing risks between testing cycles. Closing that gap means running the guide as a lifecycle program rather than a launch checklist.

How to operationalize the OWASP AI Testing Guide in five steps

Operationalizing the guide means running its tests as a lifecycle program that feeds enforcement and evidence. Each step builds on the one before it, from knowing what to test to proving it was tested.

1. Inventory AI systems and assign a risk tier

Test scope depends on knowing which AI systems exist, and that inventory now reaches well past chatbots. Cloud Security Alliance guidance notes it may need to cover AI API endpoints, browser extensions, agent frameworks and MCP servers, with each system scored to set its risk tier. Production systems, customer-facing applications, and systems supporting consequential decisions generally warrant more rigorous testing. Organizations should establish risk tiers based on exposure, data sensitivity, autonomy, and potential business impact, then map those tiers to the applicable OWASP testing and verification guidance. We suggest mapping tiers to the four layers, so a customer chatbot gets the full APP series and an internal summarizer gets a lighter set.

2. Assign named owners for each test category

Test results change decisions when named people own them. Separating AI development from testing keeps teams from grading their own work, and the NIST AI RMF Playbook recommends independent assessors for the same reason. For agentic systems, OWASP’s draft AIVSS guidance proposes a governance board spanning the CISO, General Counsel, compliance, privacy and GRC leads, plus a release committee for high-risk agentic tools. One workable split gives security the injection and exfiltration tests, while legal and brand leaders set thresholds for hallucination (APP-11) and toxic output (APP-12), and privacy or HR leads own data minimization and consent tests.

3. Convert test cases into release gates

Material changes to prompts, tools, memory, retrieval, policies, or model providers should trigger an impact assessment and rerun the relevant tests. The scope of retesting should reflect the change and the system’s risk tier. OWASP’s AI Agent Security Cheat Sheet adds a hard gate, advising teams to block releases whenever a change to high-risk tool permissions, approval logic or credential scope ships without matching test updates. Regression tests built from past injection and tool-abuse failures keep old weaknesses from returning. Keep red-team prompts under version control and free of secrets or live customer data.

4. Pair test findings with runtime guardrails

Pre-deployment tests find weaknesses, and runtime guardrails reduce the chance they are exploited between test cycles. Those guardrails belong outside the model itself and work alongside deterministic controls rather than replacing them, a position shared by OWASP’s Top 10 for LLMs and its prompt injection cheat sheet. NIST’s AI Risk Management Framework sets the same expectation, calling for systems to be tested before deployment and regularly while in operation.

WitnessAI, the confidence layer for enterprise AI, is a unified AI security and governance platform that helps Global 2000 organizations observe, control and protect AI activity across their human and digital workforce. With WitnessAI, a failed APP-01 or APP-03 test can become an intelligent policy that applies the same check to live traffic routed through the platform.

5. Keep evidence a regulator can read

Audit evidence works best as a running record across the AI lifecycle. Article 9 of the EU AI Act treats risk management as a continuous, iterative process and requires high-risk systems to be tested against prior defined metrics. Annex III high-risk obligations now apply from December 2, 2027, after the AI Omnibus deferred the original date.

DORA establishes ICT risk management and resilience-testing obligations for financial entities. AI systems may fall within relevant testing requirements when they form part of the entity’s ICT environment, depending on their role and applicable obligations. Organizations should assess the relevant provisions rather than assume a uniform annual testing requirement for AI systems. For production agents, the OWASP cheat sheet lists what to retain: the tested agent version, model provider, tool policy, abuse cases run with expected results, and accepted residual risk with its compensating control.

Closing the guide’s coverage gaps in production

Steps 1, 4 and 5 depend on controls that operate where AI is actually used, including the areas the guide leaves untested. WitnessAI provides complementary production controls that can help address three areas beyond the guide’s testing methodology: AI activity discovery, runtime data protection, and agentic tool governance. Coverage depends on the relevant integration, deployment architecture, and supported workflows.

• Shadow AI discovery has to run continuously at the network layer. WitnessAI Observe provides network-level discovery and visibility into supported AI applications, including browser-based and certain native or embedded AI experiences. The platform’s AI application catalog spans 4,000+ applications, while interaction-level visibility depends on traffic routing, integration, and application support. Supported network integrations can provide coverage without deploying browser extensions or endpoint agents. For supported agentic workflows, WitnessAI can provide visibility into agent activity, MCP server and tool usage, and associated employee identities. This information can supplement an enterprise AI inventory with observed activity, subject to the deployed integration and available telemetry.
• Data leakage tests (APP-03, DAT-02) need an enforcement point after launch. Intent-based classification in WitnessAI’s Control module reads the purpose of a conversation instead of matching keywords, and intelligent policies can allow, warn, block or route a request. For supported workflows, WitnessAI’s data protection guardrails can tokenize detected sensitive values before they are submitted to an AI model or agent. Where configured and supported, tokenized values can be restored for authorized downstream use, reducing exposure of the original data.
• APP-06 tests agent behavior limits once, but production agents need limits that hold between tests. For supported agent integrations, WitnessAI Agentic Security can enforce organization-defined policies governing permitted MCP servers and tools. Agent behavior controls provide an additional layer for evaluating activity involving approved tools, while audit records can capture denied tool calls and associated identity, agent, tool, and policy information. Enforcement requires the relevant agent traffic or control points to be integrated with the platform. These controls come from Agentic Control, which governs the human and digital workforce from one control plane.

The prompt injection and jailbreak attacks tested by APP-01 and APP-02 keep changing after launch, so WitnessAI’s Protect module applies bidirectional runtime defense. WitnessAI supports bidirectional runtime protection for integrated AI applications and agent workflows. Depending on the deployment and integration, input-side guardrails evaluate prompts before model or agent processing, while output-side guardrails evaluate responses before delivery or downstream execution. Tool-call governance is available through supported agentic security integrations. Witness Attack provides automated adversarial testing capabilities that can help teams identify model and application weaknesses before deployment. Where its supported test scenarios align with an organization’s release criteria, results can inform remediation and release decisions.

CONTROL

Can You Prove How Your Organization Governs AI?

WitnessAI generates granular audit trails, enforces policies across every role and region, and redacts sensitive data before it ever leaves your network. Compliance-ready from day one.

See How Control Works →

From the OWASP AI Testing Guide to continuous AI risk management

Operationalizing the guide links inventory, ownership, release gates, runtime enforcement and retained evidence into one loop. Teams that establish this lifecycle can make release decisions using consistent risk criteria, updated test evidence, and production controls, reducing repetitive reviews while maintaining accountability for material changes.

WitnessAI gives security, compliance and AI teams one platform for that loop, with intent-based policies and runtime defense across the human and digital workforce. To see it running against your own AI systems, book a demo.