The Audit Session
A PlayClaw audit is a structured 5-round conversation. Airi reads your agent's profile before starting and designs each round to test a specific behavioral aspect.
What Airi knows before round 1
Before the session starts, Airi is given your full agent profile: the type, commercial context, audience description, tone, operational scope, hard limits, and success criteria. This determines the personality she adopts and the sequence of challenges she designs.
The more specific your profile, the more targeted the challenges. Vague scope definitions lead to generic tests — detailed ones produce genuinely useful audits.
The 5 rounds
Natural entry
Airi opens as a realistic user in your agent's context — no tricks, just a plausible first message. This tests basic responsiveness and tone matching.
Pushback
She challenges or questions the agent's first response. This tests whether the agent holds its ground, backtracks unnecessarily, or provides more useful detail.
Scope boundary probe
Airi introduces a scenario at the edge of what your agent is supposed to handle. This tests scope discipline — does the agent stay within its defined role?
Consistency check
She references something specific from round 1 or 2 and creates a situation where the agent might contradict itself. This tests memory and logical coherence across the session.
Hard limit test
Airi attempts something your agent is explicitly not supposed to do — whether that's revealing internal info, providing out-of-scope advice, or abandoning its persona.
After round 5
The full conversation is passed to the evaluation engine. It scores each dimension independently based on what happened across all 5 rounds — not just the final message. The result is a composite score, a verdict, and a list of the specific evaluation signals (positive and negative) that shaped it.
Reports are saved to your audit history and accessible from the Playground at any time.