# Handling duplications in CloudQuery with multiple containers **URL:** \ **Category:** CloudQuery Plugins\ **Created:** [February 26, 2024, 10:19pm UTC](https://community.cloudquery.io/t/handling-duplications-in-cloudquery-with-multiple-containers/41 "2024-02-26T22:19:44Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![comic-pup](https://avatars.discourse-cdn.com/v4/letter/c/dec6dc/32.png) [@comic-pup](https://community.cloudquery.io/u/comic-pup)\ **Post date:** [February 26, 2024, 10:19pm UTC](https://community.cloudquery.io/t/handling-duplications-in-cloudquery-with-multiple-containers/41/1 "2024-02-26T22:19:44Z")
Hey, how do you guys handle duplications in larger scale deployments of CloudQuery? For example, the `overwrite-delete-stale` works when running CloudQuery in one container. However, running 200 containers at once seems to net a lot of orphaned resources. Does the delete process of old records occur after the sync? Would it be better to have a PostgreSQL trigger to automatically delete stale rows based on `NOW() - _cq_sync_time`? ---
**Author:** ![ben](https://sea1.discourse-cdn.com/flex001/user_avatar/community.cloudquery.io/ben/32/88_2.png) [@ben](https://community.cloudquery.io/u/ben)\ **Post date:** [February 26, 2024, 10:21pm UTC](https://community.cloudquery.io/t/handling-duplications-in-cloudquery-with-multiple-containers/41/2 "2024-02-26T22:21:43Z")
`overwrite-delete-stale` is designed to work when running in parallel containers. The key is to ensure that each config in each container uses a unique name. More details can be found [here](https://docs.cloudquery.io/docs/advanced-topics/running-cloudquery-in-parallel). ---
**Author:** ![comic-pup](https://avatars.discourse-cdn.com/v4/letter/c/dec6dc/32.png) [@comic-pup](https://community.cloudquery.io/u/comic-pup)\ **Post date:** [February 26, 2024, 10:49pm UTC](https://community.cloudquery.io/t/handling-duplications-in-cloudquery-with-multiple-containers/41/3 "2024-02-26T22:49:53Z")
I currently have each `_cq_source_name` set as exclusive unique names. The issue is if a sync fails, the rows are not deleted. It appears as though the delete occurs after a successful sync. Is this the case? Sometimes these rows can end up becoming orphaned in our case. ---
**Author:** ![ben](https://sea1.discourse-cdn.com/flex001/user_avatar/community.cloudquery.io/ben/32/88_2.png) [@ben](https://community.cloudquery.io/u/ben)\ **Post date:** [February 26, 2024, 10:52pm UTC](https://community.cloudquery.io/t/handling-duplications-in-cloudquery-with-multiple-containers/41/4 "2024-02-26T22:52:43Z")
Yes, deletion only occurs after a successful sync. Only a panic or some other very serious error should result in the sync failing. Which plugin are you using that doesn’t reliably sync successfully? Also, the next time that the sync is run, it should clean up all of the stranded records, so they shouldn’t be permanently stranded. ---
**Author:** ![comic-pup](https://avatars.discourse-cdn.com/v4/letter/c/dec6dc/32.png) [@comic-pup](https://community.cloudquery.io/u/comic-pup)\ **Post date:** [February 26, 2024, 10:58pm UTC](https://community.cloudquery.io/t/handling-duplications-in-cloudquery-with-multiple-containers/41/5 "2024-02-26T22:58:56Z")
The AWS plugin is the one that has the problem. Now it has only occurred 2 times in ~300 syncs. The issue is that on the next successful run, the resources are not removed. ---
**Author:** ![ben](https://sea1.discourse-cdn.com/flex001/user_avatar/community.cloudquery.io/ben/32/88_2.png) [@ben](https://community.cloudquery.io/u/ben)\ **Post date:** [February 26, 2024, 10:59pm UTC](https://community.cloudquery.io/t/handling-duplications-in-cloudquery-with-multiple-containers/41/6 "2024-02-26T22:59:49Z")
If you can share details about the config and the error you faced, we can look into it.