How We Evaluated AI Agent Security Guardrails Across Realistic Attack Scenarios

As AI agents move beyond chat and begin interacting with files, tools, APIs, and system workflows, the security problem changes. Traditional text safety still matters, but it is no longer the whole picture. Agent systems are exposed to a broader set of threats, including instruction override, indirect injection, role hijacking, tool abuse, and other attacks designed to manipulate how the agent behaves.

To better understand how current guardrail systems perform in these conditions, we conducted a comparative evaluation of DKnownAI Guard against three widely used products: AWS Bedrock Guardrails, Azure Content Safety, and Lakera Guard.

This post summarizes the motivation, methodology, and main findings from that evaluation.

Why We Ran This Evaluation

Most public safety evaluations still focus heavily on traditional harmful-content tasks. Those benchmarks are useful, but they do not fully reflect the security conditions AI agents face in production.

In real agent deployments, the attack surface is wider. Inputs may attempt to override system instructions, inject malicious directions through external content, manipulate decision logic, extract sensitive information, induce unsafe tool usage, or mix harmful-content prompts with agent-control attacks.

That means a useful agent security evaluation should test not only whether a system can block obviously harmful content, but also whether it can detect attacks against the agent itself.

What We Evaluated

We compared four products: DKnownAI Guard, AWS Bedrock Guardrails, Azure Content Safety, and Lakera Guard.

The goal was to evaluate how well each system handled two broad classes of risk: agent threats, such as instruction override, indirect injection, role hijacking, chain-of-thought poisoning, privacy leakage attempts, and tool abuse; and harmful-content elicitation, including prompts involving hate speech, pornography, and violence.

To make results comparable across systems, outputs were normalized into a binary BLOCKED / ALLOWED decision space.

Dataset and Annotation Approach

We randomly sampled 1,018 entries from eight public security datasets, including ALERT, Salad-Data, Tensor-Trust, PromptWall-Injection, CSSBench, UltraSafety, ToxicQAFinal, and an aggregated jailbreak and prompt-injection dataset.

The benchmark dataset for this evaluation is also publicly available on Hugging Face: Evaluation Dataset of AI Agent Security Guardrails.

Because many of these datasets were originally labeled for general harmfulness rather than deployment realism, we conducted additional human review on the sampled items. Each example was re-annotated as either BLOCKED if it represented a harmful or security-relevant input in a realistic deployment context, or ALLOWED if it did not present a real threat despite carrying some surface similarity to harmful samples.

• 852 entries were labeled BLOCKED
• 166 entries were labeled ALLOWED

This step matters because agent security systems are often tested on highly ambiguous boundary cases, not just obviously benign or obviously malicious content.

Main Results

Using human annotation as ground truth, we measured recall on BLOCKED samples and true negative rate on ALLOWED samples.

Within this benchmark configuration, DKnownAI Guard achieved the highest recall at 96.5% and the highest true negative rate at 90.4%, showing the strongest overall balance between detecting harmful or manipulative inputs and avoiding unnecessary blocking on difficult boundary cases. Lakera Guard also showed strong recall performance, AWS Bedrock Guardrails ranked close behind on true negative rate, and Azure Content Safety performed lower on both metrics in this setup.

Why the True Negative Rate Matters

One important takeaway from this evaluation is that false positives remain a real challenge across guardrail systems.

The ALLOWED examples in this benchmark were not ordinary benign prompts. They were boundary cases selected from predominantly harmful datasets after systematic human review. As a result, they retained partial semantic overlap with harmful content and were naturally harder to classify.

This means lower true negative rates should not be read as a simple product weakness in isolation. They also reflect the difficulty of distinguishing deceptive or ambiguous edge cases from genuinely harmful inputs.

In real deployments, overblocking can damage usability, interrupt valid workflows, and reduce trust in the system. In this setting, DKnownAI Guard's 90.4% true negative rate suggests a comparatively stronger ability to distinguish high-ambiguity borderline cases while still maintaining high recall.

Why This Matters for Agent Security

For teams shipping AI agents into real environments, the practical challenge is not just to block more attacks. It is to block attacks without breaking legitimate work.

That requires a security layer that can detect manipulation aimed at the agent itself, distinguish agent threat from harmful-content risk, handle ambiguous inputs with enough precision to avoid blunt overblocking, and support realistic enforcement decisions inside tool-using workflows.

This is the design philosophy behind DKnownAI Guard. It is built for agentic AI scenarios rather than general-purpose content filtering alone.

A Note on Methodology

This evaluation was designed to compare guardrails under a unified binary decision framework. As with any benchmark, results depend on dataset selection, annotation criteria, and output normalization choices.

We therefore view this evaluation as one useful lens on agent security performance, not as a universal ranking for every environment or use case.