lakehq/sail
Drop-in Apache Spark replacement written in Rust, unifying batch processing, stream processing, and compute-intensive AI workloads.
Build with Backblaze B2
SDKs, agent skills, IDE extensions, and reference pipelines from Backblaze Labs. All open source.
Loading star history...
Where it stands
Rank, growth pace and release cadence relative to everything GStars tracks.
3 releases in the last 90 days, the latest 28 days ago, with 52 contributors on GitHub.
Peaked at #24 in the weekly ranking the week of Apr 26, its best in the past year.
Use Cases & Benefits
- Provides a Rust-native, drop-in replacement for Apache Spark that unifies batch, stream, and AI compute workloads with Spark SQL and DataFrame API compatibility.
- Delivers up to 4x faster performance and 94% lower infrastructure costs than Spark by eliminating JVM overhead and leveraging columnar, vectorized execution with Apache Arrow.
- Use for accelerating existing PySpark workloads without code changes by connecting to Sail via the Spark Connect protocol for seamless migration.
- Use for deploying scalable, distributed data processing pipelines on Kubernetes clusters with lightweight, stateless workers optimized for cloud efficiency.
- Ideal for running compute-intensive AI and machine learning workloads alongside big data analytics using a unified, memory-safe Rust engine with zero serialization overhead.
Should you depend on it?
License terms, project hygiene and risks, from GitHub metadata and GStars' snapshots.
License
Apache 2.0
Permissive: commercial use, modification and redistribution are allowed with attribution; includes an explicit patent grant from contributors.
Community files
- README: present
- Contributing guide: present
- Code of conduct: present
Watch-outs
No watch-outs from GStars' checks.
General guidance from repository metadata, not legal advice. How we score
From the README
Sail is an open-source unified and distributed multimodal computation framework created by LakeSail.
Our mission is to unify batch processing, stream processing, and compute-intensive AI workloads. Sail is a compute engine that is:
🚀 Sail outperforms Spark, popular Spark accelerators, Databricks, and Snowflake on ClickBench.
💬 Join our Slack community to ask questions, share feedback, and connect with other Sail users and contributors.
The documentation of the latest Sail version can be found here.
Daily snapshots
Stars gained per day, and how forks and open issues moved, from GStars' daily snapshots
Forks
237+34
Open issues
325+89
Compared with peers in AI
Fastest-growing repositories GStars tracks in this cohort for the week of Nov 1, by 7-day star growth
| Repository | Stars | +7d | Health | Last release |
|---|---|---|---|---|
| thedotmack/claude-mem | 113.6K | +8.8K | 35 | today |
| D4Vinci/Scrapling | 98.1K | +6.9K | 65 | 1 day ago |
| Leonxlnx/taste-skill | 110.7K | +5.2K | 55 | 1 day ago |
| calesthio/OpenMontage | 77.2K | +4.2K | 40 | — |
| heygen-com/hyperframes | 79.6K | +4.1K | 45 | today |
| lakehq/sail | 3.5K | — | 63 | 28 days ago |
| Category median | 98.1K | +5.2K | 45 | — |
Discover Repositories
Search across tracked repositories by name or description