PeerDB-io/peerdb
Fast, Simple and a cost effective tool to replicate data from Postgres to Data Warehouses, Queues and Storage
Build with Backblaze B2
SDKs, agent skills, IDE extensions, and reference pipelines from Backblaze Labs. All open source.
Loading star history...
Where it stands
Rank, growth pace and release cadence relative to everything GStars tracks.
5 releases in the last 90 days, the latest 1 day ago, with 63 contributors on GitHub.
Use Cases & Benefits
- Provides a fast, simple, and cost-effective tool to replicate data from Postgres to data warehouses, queues, and storage systems.
- Delivers 10x faster change data capture with native Postgres optimizations and supports complex data types and schema changes reliably.
- Use for real-time streaming of Postgres data into cloud data warehouses like BigQuery or Snowflake for analytics.
- Use for building fault-tolerant ETL pipelines that handle large tables with parallelized initial loads and automatic retries.
- Use for integrating Postgres with event streaming platforms like Kafka or Azure Event Hubs using native CDC and cursor-based modes.
Should you depend on it?
License terms, project hygiene and risks, from GitHub metadata and GStars' snapshots.
License
AGPL 3.0
Network copyleft: like the GPL, and also triggered by offering the software over a network.
Community files
- README: present
- Contributing guide: present
Watch-outs
- AGPL-style copyleft also applies when the software is offered over a network.
General guidance from repository metadata, not legal advice. How we score
From the README
At PeerDB, we are building a fast, simple and the most cost effective way to stream data from Postgres to Data Warehouses, Queues and Storage engines. If you are running Postgres at the heart of your data-stack and move data at scale from Postgres to any of the above targets, PeerDB can provide value.
We support different modes of streaming - log based (CDC), cursor based (timestamp or integer) and XMIN based. Performance wise, we are 10x faster than existing tools. Features wise, we support native Postgres features such as comprehensive set of data-types incl. jsonb/arrays/geospatial, efficiently streaming TOAST columns, schema changes and so on.
If your ClickHouse DB runs outside Docker (e.g., on VMs or ClickHouse Cloud), it may not have access to MinIO, which is used by PeerDB internally to stage files before loading them. Ensure ClickHouse has network access to MinIO.
PeerDB stages PostgreSQL data in MinIO within the Docker stack. Since ClickHouse is outside Docker, it needs a resolvable hostname for MinIO.
Update
docker-compose.ymland setAWS_ENDPOINT_URL_S3to MinIO's accessible IP (from both PeerDB and ClickHouse):Rerun Docker Compose to apply changes. On AWS/GCP/Azure, also ensure the security group allows inbound access to MinIO.
Follow this 5-minute Quickstart Guide to see PeerDB in action i.e. streaming data in real-time across stores.
Daily snapshots
Stars gained per day, and how forks and open issues moved, from GStars' daily snapshots
Forks
230+31
Open issues
211+15
Compared with peers in Cloud
Fastest-growing repositories GStars tracks in this cohort for the week of Nov 1, by 7-day star growth
| Repository | Stars | +7d | Health | Last release |
|---|---|---|---|---|
| Asymptote-Labs/agent-beacon | 3.3K | +360 | 40 | today |
| juicedata/juicefs | 14.7K | +105 | 64 | 14 days ago |
| GoogleCloudPlatform/microservices-demo | 21.1K | — | 63 | 38 days ago |
| quarkusio/quarkus | 16.0K | — | 60 | 2 days ago |
| awslabs/mcp | 9.8K | — | 65 | 9 days ago |
| PeerDB-io/peerdb | 3.3K | — | 60 | 1 day ago |
| Category median | 14.7K | +233 | 63 | — |
Discover Repositories
Search across tracked repositories by name or description