gunnarmorling/1brc
1️⃣🐝🏎️ The One Billion Row Challenge -- A fun exploration of how quickly 1B rows from a text file can be aggregated with Java
Build with Backblaze B2
SDKs, agent skills, IDE extensions, and reference pipelines from Backblaze Labs. All open source.
Loading star history...
Where it stands
Rank, growth pace and release cadence relative to everything GStars tracks.
The project lists 177 contributors on GitHub.
Peaked at #83 in the weekly ranking the week of Aug 31, its best in the past year.
Use Cases & Benefits
- Explores high-performance aggregation of one billion rows from a text file using Java to compute min, mean, and max temperatures per weather station.
- Leverages advanced Java optimizations including multi-threading, SIMD, and Unsafe operations to achieve extremely fast processing times.
- Use for benchmarking and optimizing Java code performance on large-scale data aggregation tasks.
- Use for learning advanced Java techniques like GraalVM native image compilation and memory management optimizations.
- Use for comparing different Java implementations and approaches to efficiently process massive text-based datasets.
Should you depend on it?
License terms, project hygiene and risks, from GitHub metadata and GStars' snapshots.
License
Apache 2.0
Permissive: commercial use, modification and redistribution are allowed with attribution; includes an explicit patent grant from contributors.
Community files
- README: present
Watch-outs
- Warning: No commits in over a year (last push Aug 2024).
- No GitHub releases; pin a commit or tag instead of a release.
General guidance from repository metadata, not legal advice. How we score
From the README
Status Feb 4: The final leaderboards have been published. Congrats to all the winners, and a big thank you to everyone participating in this challenge as well as to everyone helping to organize it!
Status Feb 3: All entries have been evaluated and I am in the process of finalizing the leaderboards.
Status Feb 1: The challenge has been closed for new submissions. No new pull requests for adding submissions are accepted at this time. Pending PRs will be evaluated over the next few days.
Status Jan 31: The challenge will close today at midnight UTC.
Status Jan 12: As there has been such a large number of entries to this challenge so far (100+), and this is becoming hard to manage, please only create new submissions if you expect them to run in 10 seconds or less on the evaluation machine.
Status Jan 1: This challenge is open for submissions!
Sponsorship
A big thank you to my employer Decodable for funding the evaluation environment and supporting this challenge!
The One Billion Row Challenge (1BRC) is a fun exploration of how far modern Java can be pushed for aggregating one billion rows from a text file. Grab all your (virtual) threads, reach out to SIMD, optimize your GC, or pull any other trick, and create the fastest implementation for solving this task!
Daily snapshots
Stars gained per day, and how forks and open issues moved, from GStars' daily snapshots
Forks
2.2K+3
Open issues
40-1
Compared with peers in Backend
Fastest-growing repositories GStars tracks in this cohort for the week of Nov 8, by 7-day star growth
| Repository | Stars | +7d | Health | Last release |
|---|---|---|---|---|
| anthropics/skills | 192.3K | +5.2K | 75 | — |
| zed-industries/zed | 94.7K | +2.3K | 75 | 6 days ago |
| google/skills | 23.8K | +2.0K | 75 | — |
| NationalSecurityAgency/ghidra | 99.4K | +1.8K | 75 | 19 days ago |
| DuarteSantos8/openGym | 20.8K | +1.7K | 75 | 7 days ago |
| gunnarmorling/1brc | 8.1K | — | 35 | — |
| Category median | 94.7K | +2.0K | 75 | — |
Discover Repositories
Search across tracked repositories by name or description