Syncing large amounts of cloud data (especially from AWS) can be a time-consuming process. But with CloudQuery’s new Automatic Sharding feature, you can now split your syncs into multiple shards and run them in parallel, reducing sync time drastically! ![]()
What’s Automatic Sharding?
Sharding allows CloudQuery to split the collection of tables into groups and sync them independently. Simply use the --shard option to define how many shards you’d like to use, and CloudQuery will handle the rest!
Real-World Example:
Setting up a parallel sync with GitHub Actions is now easier than ever. Just set up a matrix configuration to split your syncs into multiple jobs and run them concurrently. ![]()
Check out the full post and documentation to see how you can leverage Automatic Sharding and make your syncs faster than ever.