Error syncing v3 source with CloudQuery EOF issue

[devoted-oryx]

Hi CQ team,

I’m receiving the following error:

Error: failed to sync v3 source
2024-01-18T13:53:27Z ERR exiting with error error="failed to sync v3 source Account1: write client returned error (insert): error reading from server: EOF" module=cli

Here is my config:

path: cloudquery/aws
version: "v23.5.0"
tables:
  - aws_securityhub_findings
destinations: ["postgresql"]

Can anyone help me troubleshoot this issue?

[ben]

Does this same error happen every time you run a sync? What happens if you sync another table, say, aws_lambda_runtimes?

[devoted-oryx]

Hi cq-ben,

It seems it’s related to the memory limit.

[ben]

Interesting! How do you determine that? How many accounts + regions are you syncing?

[devoted-oryx]

So basically, I have a central security hub that aggregates findings from all accounts. The accounts count is around 50. I increased the pod memory limit from 1GB to 2GB and now it’s syncing properly (at least it looks like that).