sierra-research/tau2-bench
τ-Bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains
Build with Backblaze B2
SDKs, agent skills, IDE extensions, and reference pipelines from Backblaze Labs. All open source.
Loading star history...
Where it stands
Rank, growth pace and release cadence relative to everything GStars tracks.
1 release in the last 90 days, the latest 15 days ago, with 20 contributors on GitHub.
Use Cases & Benefits
- Implements a simulation framework to benchmark conversational agents interacting with users and tools across real-world domains.
- Provides a standardized, gym-compatible environment enabling consistent training, evaluation, and comparison of language model agents.
- Use for training reinforcement learning agents on multi-domain customer service tasks with standardized train/test splits.
- Use for interactively playing as agent or user to debug, test strategies, and understand domain policies and tools.
- Use for submitting and comparing agent performance on a live leaderboard covering airline, retail, and telecom domains.
Should you depend on it?
License terms, project hygiene and risks, from GitHub metadata and GStars' snapshots.
License
MIT
Permissive: commercial use, modification and redistribution are allowed with attribution.
Community files
- README: present
- Contributing guide: present
Watch-outs
No watch-outs from GStars' checks.
General guidance from repository metadata, not legal advice. How we score
From the README
Figure 1: τ²-bench allows users to interact with the agent and the environment
Figure 2: Trajectory of a conversation between an agent and a user
τ²-bench now supports RL training with a Gymnasium-compatible interface:
⚠️ IMPORTANT FOR BACKWARD COMPATIBILITY: If you are just evaluating an agent (not training), you MUST use the
basetask split to evaluate on the complete task set that matches the original τ²-bench structure. This ensures your results are comparable to previous evaluations and maintains consistency with the established benchmark. (If you don't specify a task split, it will default tobase.)
Daily snapshots
Stars gained per day, and how forks and open issues moved, from GStars' daily snapshots
Forks
624+190
Open issues
340+186
Compared with peers in AI
Fastest-growing repositories GStars tracks in this cohort for the week of Nov 1, by 7-day star growth
| Repository | Stars | +7d | Health | Last release |
|---|---|---|---|---|
| thedotmack/claude-mem | 113.6K | +8.8K | 35 | today |
| D4Vinci/Scrapling | 98.1K | +6.9K | 65 | 1 day ago |
| Leonxlnx/taste-skill | 110.7K | +5.2K | 55 | 1 day ago |
| calesthio/OpenMontage | 77.2K | +4.2K | 40 | — |
| heygen-com/hyperframes | 79.6K | +4.1K | 45 | today |
| sierra-research/tau2-bench | 2.4K | — | 59 | 15 days ago |
| Category median | 98.1K | +5.2K | 45 | — |
Discover Repositories
Search across tracked repositories by name or description