harveyai/harvey-labs
A benchmark built to evaluate and improve agent capabilities for supporting legal work.
Build with Backblaze B2
SDKs, agent skills, IDE extensions, and reference pipelines from Backblaze Labs. All open source.
Loading star history...
Where it stands
Rank, growth pace and release cadence relative to everything GStars tracks.
3 releases in the last 90 days, the latest 11 days ago, with 22 contributors on GitHub.
Use Cases & Benefits
- Provides an open-source benchmark to evaluate and improve AI agents' capabilities in performing realistic legal work tasks.
- Enables standardized, reproducible assessment of legal AI agents using a comprehensive dataset and execution harness.
- Use for testing AI agents on complex legal tasks such as M&A data-room assignments with detailed scoring and reporting.
- Use for developing and refining AI models to handle diverse legal practice areas and contracting scenarios.
- Use for academic research to benchmark and compare legal AI agent performance with transparent evaluation methodologies.
Should you depend on it?
License terms, project hygiene and risks, from GitHub metadata and GStars' snapshots.
License
MIT
Permissive: commercial use, modification and redistribution are allowed with attribution.
Community files
- README: present
- Contributing guide: present
Watch-outs
No watch-outs from GStars' checks.
General guidance from repository metadata, not legal advice. How we score
From the README
Legal Agent Benchmark (LAB): An open-source benchmark for evaluating agents on real legal work.
Harvey LAB is an open-source project aimed at benchmarking LLM agents' abilities to perform legal work in realistic environments.
LAB consists of two parts: a dataset of tasks containing agent instructions, documents, and rubrics as well as an execution harness for running and evaluating agents against those tasks.
LAB is an ongoing project and we expect to consistently add to and refine the task set and execution harness.
Read the announcement post: Introducing Harvey's Legal Agent Benchmark
Start with the full walkthrough in docs/tutorial.md — it takes one realistic M&A data-room assignment end to end: setup, task inspection, agent run, scoring, report review, and comparison dashboards.
Daily snapshots
Stars gained per day, and how forks and open issues moved, from GStars' daily snapshots
+889 stars over 86 days across 14 snapshot days; best day +139 on Aug 11.
Forks
290+124
Open issues
64+18
Compared with peers in Backend
Fastest-growing repositories GStars tracks in this cohort for the week of Nov 1, by 7-day star growth
| Repository | Stars | +7d | Health | Last release |
|---|---|---|---|---|
| anthropics/skills | 187.8K | +2.9K | 15 | — |
| DuarteSantos8/openGym | 19.7K | +2.4K | 60 | 2 days ago |
| shiyu-coder/Kronos | 47.0K | +2.3K | 16 | — |
| NationalSecurityAgency/ghidra | 98.2K | +2.3K | 64 | 14 days ago |
| microsoft/VibeVoice | 56.0K | +1.1K | 29 | — |
| harveyai/harvey-labs | 1.5K | — | 45 | 11 days ago |
| Category median | 56.0K | +2.3K | 29 | — |
Discover Repositories
Search across tracked repositories by name or description