vortex-data/vortex
An extensible, state-of-the-art framework for columnar compression, and the fastest FOSS columnar file format. Formerly at @spiraldb, now an Incubation Stage project at LFAI&Data, part of the Linux Foundation.
Build with Backblaze B2
SDKs, agent skills, IDE extensions, and reference pipelines from Backblaze Labs. All open source.
Loading star history...
Where it stands
Rank, growth pace and release cadence relative to everything GStars tracks.
5 releases in the last 90 days, the latest 3 days ago, with 92 contributors on GitHub.
Peaked at #34 in the weekly ranking the week of Dec 7, its best in the past year.
Use Cases & Benefits
- Provides a high-performance, extensible columnar file format and toolkit optimized for fast data processing and compression.
- Delivers blazing fast random access, scan, and write speeds with extensible encoding and zero-copy Apache Arrow integration.
- Use for building scalable data systems requiring efficient columnar storage with fast read/write access on object storage.
- Use for integrating with data processing frameworks like Spark, DuckDB, and Pandas to accelerate analytics workflows.
- Use for developing custom compression schemes or encoding strategies leveraging its pluggable architecture and Rust implementation.
Should you depend on it?
License terms, project hygiene and risks, from GitHub metadata and GStars' snapshots.
License
Apache 2.0
Permissive: commercial use, modification and redistribution are allowed with attribution; includes an explicit patent grant from contributors.
Community files
- README: present
- Contributing guide: present
Watch-outs
No watch-outs from GStars' checks.
General guidance from repository metadata, not legal advice. How we score
From the README
Vortex is a next-generation columnar file format and toolkit designed for high-performance data processing. It is the fastest and most extensible format for building data systems backed by object storage. It provides:
🟢 Development Status: Library APIs may change from version to version, but we now consider the file format stable. From release 0.36.0, all future releases of Vortex should maintain backwards compatibility of the file format (i.e., be able to read files written by any earlier version >= 0.36.0).
Daily snapshots
Stars gained per day, and how forks and open issues moved, from GStars' daily snapshots
Forks
253+60
Open issues
428+124
Compared with peers in Backend
Fastest-growing repositories GStars tracks in this cohort for the week of Nov 1, by 7-day star growth
| Repository | Stars | +7d | Health | Last release |
|---|---|---|---|---|
| anthropics/skills | 187.8K | +2.9K | 15 | — |
| DuarteSantos8/openGym | 19.7K | +2.4K | 60 | 2 days ago |
| shiyu-coder/Kronos | 47.0K | +2.3K | 16 | — |
| NationalSecurityAgency/ghidra | 98.2K | +2.3K | 64 | 14 days ago |
| microsoft/VibeVoice | 56.0K | +1.1K | 29 | — |
| vortex-data/vortex | 3.3K | — | 60 | 3 days ago |
| Category median | 56.0K | +2.3K | 29 | — |
Discover Repositories
Search across tracked repositories by name or description