Also, failed to sync v3 source aws_xxxxxxxxxxxxx: write client returned error (insert): plugin returned error: failed to add column resolver_count_limit on table aws_appsync_graphql_apis:
ERROR: column "resolver_count_limit" of relation "aws_appsync_graphql_apis" already exists (SQLSTATE 42701)
It could also be the PG plugin that needs to be updated as well (if using that).
But I remember seeing the issue with columns and wondering why migration wasn’t working. If using the PostgREST plugin, you may need to force migration.
So it happened right around this time: link
Not the same error in the thread, but I saw it around that time. Updating fixed it, if I remember correctly.
Also, for these versions, I am running into an issue where the syncing of tables is taking longer than expected. For the previous set of versions, it was good, and I was able to run it for around 1100 accounts in 2 and a half hours. But now it is taking around 10 hours. I have also skipped the aws_inspector_findings and aws_inspector2_findings tables.
Can I please know to which versions I should upgrade so that I can have a good migration?
Can you share the configuration spec you’re using to run the syncs? Each new plugin version adds new tables, so you might be syncing more data than before.
For the already exists (SQLSTATE 42701) error, are you running multiple syncs at the same time?
To best assist you as we work through this issue, I am interested to learn how you and your organization are using CloudQuery. I would love to hop on a call to discuss. Feel free to schedule a slot on my calendar: https://calendar.app.google/tYWo4ox9GwwMAuUR8
Can you suggest the recommended tables to skip for AWS plugin 23.1.0? Also, could you provide the required PostgreSQL and CLI plugins for this AWS plugin version?