Ultra13
Platform · Benchmarks

How does Ultra13 benchmark AI agent security?

Ultra13 benchmarks agent security by replaying attack families against a workflow with the firewall off and then on. A finding counts as neutralized only when the unprotected run succeeds and the protected run blocks, redacts, quarantines, or gates the path before it reaches the unsafe sink.

By the Ultra13 teamPublished Updated
TL;DR
  • Attack families are replayed OFF then ON.
  • Neutralized = succeeds unprotected AND blocked when protected.
  • Coverage is tied to product/policy version, models, and date.
  • Full methodology and limitations are published separately.

Read the methodology

Headline coverage numbers only mean something with a methodology behind them. The full definitions, attack-family table, corpora, and limitations live on the benchmark methodology page, with a worked single-finding example in the sample proof report.

FAQ

Frequently asked questions

Do benchmark results apply to my deployment?
Coverage depends on your workflow, tools, data classes, policy configuration, deployment mode, and validation scope. Numbers are tied to a specific product and policy version and test date.

See where the firewall stops the path.

Give us one agent workflow. We’ll map the context boundary, replay the abuse paths, and show where the Context Firewall blocks them.

Join WaitlistSee a Sample Report