huggingface/xet-core
xet client tech, used in huggingface_hub
Build with Backblaze B2
SDKs, agent skills, IDE extensions, and reference pipelines from Backblaze Labs. All open source.
Loading star history...
Where it stands
Rank, growth pace and release cadence relative to everything GStars tracks.
4 releases in the last 90 days, the latest today, with 31 contributors on GitHub.
Gained 9% more stars in the last 90 days than in the 90 days before.
Use Cases & Benefits
- Implements a Rust-based client library enabling chunk-based deduplication and efficient storage for uploading and downloading data to the HuggingFace Hub.
- Provides optimized storage and retrieval with local disk caching and backwards compatibility with Git LFS, reducing redundant data transfer and storage.
- Use for integrating efficient chunk deduplication in machine learning model and dataset storage workflows on HuggingFace Hub.
- Use for developing Python applications that require performant and concurrent network communication with HuggingFace Hub backend services via Python bindings.
- Use for debugging and diagnosing issues in data transfer processes with platform-specific diagnostic tools and debug symbol support.
Should you depend on it?
License terms, project hygiene and risks, from GitHub metadata and GStars' snapshots.
License
Apache 2.0
Permissive: commercial use, modification and redistribution are allowed with attribution; includes an explicit patent grant from contributors.
Community files
- README: present
- Contributing guide: present
- Code of conduct: present
Watch-outs
No watch-outs from GStars' checks.
General guidance from repository metadata, not legal advice. How we score
From the README
🤗 xet-core - xet client tech, used in huggingface_hub
xet-core enables huggingface_hub to utilize xet storage for uploading and downloading to HF Hub. Xet storage provides chunk-based deduplication, efficient storage/retrieval with local disk caching, and backwards compatibility with Git LFS. This library is not meant to be used directly, and is instead intended to be used from huggingface_hub.
♻ chunk-based deduplication implementation: avoid transferring and storing chunks that are shared across binary files (models, datasets, etc).
🤗 Python bindings: bindings for huggingface_hub package.
↔ network communications: concurrent communication to HF Hub Xet backend services (CAS).
🔖 local disk caching: chunk-based cache that sits alongside the existing huggingface_hub disk cache.
Daily snapshots
Stars gained per day, and how forks and open issues moved, from GStars' daily snapshots
Forks
111+26
Open issues
63+31
Compared with peers in Backend
Fastest-growing repositories GStars tracks in this cohort for the week of Nov 1, by 7-day star growth
| Repository | Stars | +7d | Health | Last release |
|---|---|---|---|---|
| anthropics/skills | 187.8K | +2.9K | 15 | — |
| DuarteSantos8/openGym | 19.7K | +2.4K | 60 | 2 days ago |
| shiyu-coder/Kronos | 47.0K | +2.3K | 16 | — |
| NationalSecurityAgency/ghidra | 98.2K | +2.3K | 64 | 14 days ago |
| microsoft/VibeVoice | 56.0K | +1.1K | 29 | — |
| huggingface/xet-core | 622 | — | 39 | today |
| Category median | 56.0K | +2.3K | 29 | — |
Also by huggingface
Other repositories from this maintainer that GStars tracks
- Python168.6K stars
transformers
🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.
- Python37.3K stars
pytorch-image-models
The largest collection of PyTorch image encoders / backbones. Including train, eval, inference, export scripts, and pretrained weights -- ResNet, ResNeXT, EfficientNet, NFNet, Vision Transformer (ViT), MobileNetV4, MobileNet-V3 & V2, RegNet, DPN, CSPNet, Swin Transformer, MaxViT, CoAtNet, ConvNeXt, and more
- Python34.8K stars
diffusers
🤗 Diffusers: State-of-the-art diffusion models for image, video, and audio generation in PyTorch.
- Python28.7K stars
lerobot
🤗 LeRobot: Making AI for Robotics more accessible with end-to-end learning
- Python22.1K stars
datasets
🤗 The largest hub of ready-to-use datasets for AI models with fast, easy-to-use and efficient data manipulation tools
Discover Repositories
Search across tracked repositories by name or description