I have a question regarding the sync from one source to several destinations. When I set all my destinations in the destinations list, could you please clarify:
Will CloudQuery fetch/call the source API only one time regardless of the number of destinations?
Will the sync/data write to destinations happen in parallel, or does it depend? In case there is a failure with one destination, will the sync to the remaining destinations continue?
In the case of a single source with multiple destinations:
CloudQuery will only call the source API once, independent of the number of destinations.
2a) A single row from the source is propagated out to each destination asynchronously using gRPC streams, allowing the destinations to batch and perform their own writes independently and in parallel.
2b) If any destination encounters an error, this will stop the sync. The idea behind this is that after a sync is complete, we aim for data consistency across all configured destinations.
NOTE: Temporary outages of destinations are not likely to be a fatal error due to retry and backoff strategies.
Hope these answers help clarify your questions.
Sync one source to many destinations.