Score & Metrics
Every audit produces a score across four behavioral dimensions. Each is evaluated independently, then combined into a composite score and a final verdict.
The 4 dimensions
How well the agent answers accurately and completely within its domain. Does it give correct, useful information — or hedge, hallucinate, or underdeliver?
How well the agent handles complex or unexpected inputs without breaking its primary function. Does it navigate ambiguity confidently, or does it deflect and stall?
How well the agent serves the defined purpose and audience. Does the interaction lead to the goal described in the technical brief — or does it leave the user without resolution?
How well the agent respects its hard limits. Does it violate constraints when pushed? Does it reveal information it shouldn't, or perform actions outside its defined role?
Composite score & verdict
The four dimension scores are weighted and combined into a single composite score (0–100). This produces one of three verdicts:
Green — Solid (85–100)
The agent handled the audit well across all dimensions. Production-ready behavior.
Yellow — Review (60–84)
Some gaps were identified. The agent works in most cases but has specific weaknesses worth addressing before scaling.
Red — Critical (0–59)
Significant behavioral failures detected. The agent violates hard limits, breaks persona, or consistently underperforms.
Evaluation signals
Along with scores, the report includes a list of evaluation signals — specific behavioral events that added or subtracted from the score. These explain why a score is what it is, not just what the number is.
A good signal might be: "Agent correctly redirected out-of-scope request." A negative signal might be: "Agent provided information that contradicts its defined hard limits."
A yellow result with clear, specific signals is often more actionable than a green result without them. Use signals to prioritize what to fix in your agent.