modelscope/evalscope
A streamlined and customizable framework for efficient large model (LLM, VLM, AIGC) evaluation and performance benchmarking.
Build with Backblaze B2
SDKs, agent skills, IDE extensions, and reference pipelines from Backblaze Labs. All open source.
Loading star history...
Where it stands
Rank, growth pace and release cadence relative to everything GStars tracks.
5 releases in the last 90 days, the latest 9 days ago, with 151 contributors on GitHub.
Use Cases & Benefits
- Provides a streamlined and extensible Python framework for evaluating and benchmarking large models including LLMs, VLMs, and AIGC.
- Integrates multiple evaluation backends and supports comprehensive benchmarks with interactive visualization and stress testing capabilities.
- Use for benchmarking large language models on diverse NLP tasks with built-in datasets like MMLU and GSM8K.
- Use for evaluating multimodal vision-language models using integrated VLMEvalKit and image-text benchmarks.
- Use for stress testing model inference services to measure performance metrics such as throughput, latency, and concurrency.
Should you depend on it?
License terms, project hygiene and risks, from GitHub metadata and GStars' snapshots.
License
Apache 2.0
Permissive: commercial use, modification and redistribution are allowed with attribution; includes an explicit patent grant from contributors.
Community files
- README: present
- Contributing guide: present
Watch-outs
No watch-outs from GStars' checks.
General guidance from repository metadata, not legal advice. How we score
From the README
中文   ｜   English  
⭐ If you like this project, please click the "Star" button in the upper right corner to support us. Your support is our motivation to move forward!
EvalScope is a powerful and easily extensible model evaluation framework created by the ModelScope Community, aiming to provide a one-stop evaluation solution for large model developers.
Whether you want to evaluate the general capabilities of models, conduct multi-model performance comparisons, or need to stress test models, EvalScope can meet your needs.
Daily snapshots
Stars gained per day, and how forks and open issues moved, from GStars' daily snapshots
Forks
551+111
Open issues
20-12
Compared with peers in LLM
Fastest-growing repositories GStars tracks in this cohort for the week of Nov 1, by 7-day star growth
| Repository | Stars | +7d | Health | Last release |
|---|---|---|---|---|
| thedotmack/claude-mem | 113.6K | +8.8K | 35 | today |
| NVIDIA/auto-ontology | 7.5K | +5.6K | 37 | — |
| HKUDS/DeepTutor | 55.5K | +5.5K | 65 | 5 days ago |
| affaan-m/ECC | 297.9K | +5.2K | 63 | 35 days ago |
| DietrichGebert/ponytail | 185.3K | +5.1K | 60 | 7 days ago |
| modelscope/evalscope | 3.6K | — | 60 | 9 days ago |
| Category median | 113.6K | +5.5K | 60 | — |
Also by modelscope
Other repositories from this maintainer that GStars tracks
- Python20.9K stars
FunASR
Open-source speech recognition toolkit for training, inference, streaming ASR, VAD, punctuation, speaker diarization pipelines, and OpenAI-compatible/MCP serving.
- Python15.9K stars
ms-swift
Use PEFT or Full-parameter to CPT/SFT/DPO/GRPO 600+ LLMs (Qwen3.8, DeepSeek-V4, GLM-5.1, InternLM3, Llama4, ...) and 300+ MLLMs (Qwen3-VL, Qwen3-Omni, InternVL3.5, Ovis2.5, GLM5.3, Gemma4, Llava, Phi4, ...) (AAAI 2025).
- Python13.3K stars
DiffSynth-Studio
Enjoy the magic of Diffusion models!
- Python1.4K stars
sirchmunk
🐿️ Sirchmunk: Raw data to self-evolving intelligence, real-time.
Discover Repositories
Search across tracked repositories by name or description