Agent Evaluation
Multi-turn trajectory testing for tool-using AI agents. Measure planning validity, API call schema compliance, environment state transitions, and error recovery behavior under simulated failure conditions.
What We Evaluate
Single-turn prompt benchmarks cannot predict whether an agent will succeed when executing a 12-step workflow across external databases and APIs. We test multi-step decision trees, state persistence, token efficiency across long trajectories, and whether the agent detects when a sub-task has failed.
Fault Injection & Recovery Loops
Real environments fail: APIs return rate limits (429), microservices throw 503s, and network sockets time out. We inject controlled fault states into sandboxed environments to verify whether your agent backs off gracefully or enters infinite, token-wasting retry loops.
| Metric | Dimension | Description |
|---|---|---|
| End-to-End Task Completion | GOAL PASS | Percentage of multi-step workflows where the environment reaches the verified terminal goal state. |
| Tool Call Schema Validity | SYNTAX RIGOR | Rate of valid JSON/RPC tool invocations matching defined OpenAPI/Pydantic schemas. |
| Transient Fault Recovery Rate | RESILIENCE | Success rate when recovering from injected simulated server errors (HTTP 429, 503, timeouts). |
| Infinite Loop Incidence | STABILITY | Rate of trajectories entering repetitive circular execution loops with identical arguments. |
| Injected Fault Scenario | Runs | Recovery Rate | Loop Incidence | Token Overhead |
|---|---|---|---|---|
| Nominal Happy-Path Workflow | 250 | 94.8% | 0.0% | 1.0x (Baseline) |
| Transient Rate Limit (HTTP 429) | 100 | 81.0% | 6.0% | 1.4x |
| Service Unavailable (HTTP 503) | 100 | 24.0% | 38.0% | 3.4x |
Benchmark Your Autonomous Agents
Test multi-turn tool execution, error handling, and trajectory stability in sandboxed environments under mutual NDA.