Tau-squared Bench — evaluates conversational agent capabilities including multi-turn reasoning, tool use, and maintaining coherent long interactions.
Evaluates multi-turn tool use and conversational state tracking, which are under-measured by single-turn QA benchmarks; scenario framing helps expose degradation across longer interaction horizons.
Scenarios are scripted compared with production traffic, so behavior under adversarial users can differ; rubric-driven success metrics may miss subtle quality failures between turns.
Higher is better. Measures success rate on multi-turn conversational tasks.
Max score: 100
Based on ~200 samples, ~2,000 input tokens and ~2,000 output tokens per sample (4,000 total per sample).
Estimates from τ²-Bench paper. Actual costs vary with reasoning effort and multi-turn prompting.
Scenarios are scripted compared with production traffic, so behavior under adversarial users can differ.
| # | Model | Score | Est. Eval Cost | Score/$ |
|---|---|---|---|---|
| 1 | 99.1 | $2.32 | 42.7 | |
| 2 | 99.1 | $0.0000 | — | |
| 3 | 98.8 | $0.19 | 526 | |
| 4 | 98.5 | $0.0000 | — | |
| 5 | 98.5 | $0.0000 | — | |
| 6 | 98.5 | $0.54 | 182 | |
| 7 | 98.5 | $24.00 | 4.11 | |
| 8 | 98.5 | $2.08 | 47.4 | |
| 9 | 98.5 | $1.66 | 59.2 | |
| 10 | 98.2 | $1.68 | 58.5 | |
| 11 | 97.7 | $1.40 | 69.8 | |
| 12 | 97.7 | $1.97 | 49.5 | |
| 13 | 97.7 | $1.50 | 65.1 | |
| 14 | 97.4 | $1.68 | 58.0 | |
| 15 | 97.1 | $0.0000 | — | |
| 16 | 97.1 | $2.14 | 45.3 | |
| 17 | 96.5 | $3.20 | 30.2 | |
| 18 | 96.2 | $1.78 | 54.1 | |
| 19 | 95.9 | $1.44 | 66.6 | |
| 20 | 95.9 | $1.05 | 91.3 | |
| 21 | 95.9 | $0.0000 | — | |
| 22 | 95.9 | $1.98 | 48.4 | |
| 23 | 95.9 | $3.64 | 26.3 | |
| 24 | 95.6 | $1.68 | 56.9 | |
| 25 | 95.6 | $4.20 | 22.8 | |
| 26 | 95.6 | $5.60 | 17.1 | |
| 27 | 95.6 | $0.14 | 691 | |
| 28 | 95.6 | $1.68 | 56.9 | |
| 29 | 95.3 | $4.20 | 22.7 | |
| 30 | 95.3 | $0.60 | 159 | |
| 31 | 95.3 | $1.05 | 90.8 | |
| 32 | 95.0 | $0.16 | 594 | |
| 33 | 95.0 | $0.0000 | — | |
| 34 | 95.0 | $0.14 | 687 | |
| 35 | 94.7 | $4.00 | 23.7 | |
| 36 | 94.4 | $0.16 | 590 | |
| 37 | 94.4 | $12.00 | 7.87 | |
| 38 | 94.4 | $0.11 | 837 | |
| 39 | 94.2 | $1.92 | 49.0 | |
| 40 | 94.2 | $1.05 | 89.7 | |
| 41 | 94.2 | $1.68 | 56.0 | |
| 42 | 94.2 | $0.52 | 180 | |
| 43 | 94.2 | $3.60 | 26.2 | |
| 44 | 93.9 | $1.08 | 86.9 | |
| 45 | 93.9 | $14.00 | 6.70 | |
| 46 | 93.9 | $1.98 | 47.4 | |
| 47 | 93.6 | $1.44 | 65.0 | |
| 48 | 93.6 | $1.68 | 55.7 | |
| 49 | 93.3 | $0.0000 | — | |
| 50 | 93.3 | $0.0000 | — |