Error fetching plugin metadata from CloudQuery due to connection reset

CloudQuery Plugins
1

Hi team,

I am facing this problem:

Error: failed to get plugin metadata from hub: failed to request: GET "https://api.cloudquery.io/plugins/cloudquery/AWS/versions/v23.5.0/assets/linux_amd64": read: connection reset by peer

Can anyone help me troubleshoot this issue?

2

Hi @sweeping-caribou :wave:

Where are you running CloudQuery and encountering this error? Is it from within your cloud provider? Any details you can provide will help:

  • The country/region this is running from,
  • The cloud provider being used,
  • Any network firewalls?
  • The (redacted) CloudQuery config you are using,
  • Does it always fail, or only sometimes?
3

Hiii @herman,

I am running this in one VM. I am following the below document to download CloudQuery:

curl -L https://github.com/cloudquery/cloudquery/releases/download/cli-v5.0.0/cloudquery_linux_amd64 -o cloudquery
chmod a+x cloudquery

Here is my configuration for the source plugin:

kind: source
spec:
  name: aws
  path: cloudquery/aws
  registry: cloudquery
  version: "v23.5.0"
  tables: ['aws_s3_buckets']
  destinations: ["postgresql"]
  spec:
    aws_debug: false
    org:
      admin_account:
        local_profile: "<NAMED_PROFILE>"
      member_role_name: OrganizationAccountAccessRole
    regions:
      - '*'
    # Optional parameters
    # regions: []
    # accounts: []
    # org: nil
    # concurrency: 50000
    # initialization_concurrency: 4
    # aws_debug: false
    # max_retries: 10
    # max_backoff: 30
    # custom_endpoint_url: ""
    # custom_endpoint_hostname_immutable: nil # required when custom_endpoint_url is set
    # custom_endpoint_partition_id: "" # required when custom_endpoint_url is set
    # custom_endpoint_signing_region: "" # required when custom_endpoint_url is set
    # use_paid_apis: false
    # table_options: nil
    # scheduler: dfs # options are: dfs, round-robin or shuffle

I am using this source plugin… My old plugins are completely working fine, but the new one is throwing an error.

4

Which cloud and country/region is this running in? Are there any network firewalls that might prevent connecting to https://api.cloudquery.io?

I don’t know your previous config, but it may have been using the GitHub registry, which downloaded plugins from github.com. Since the launch of the CloudQuery Hub, plugins are now downloaded via the API at https://api.cloudquery.io, so you should make sure that this is reachable from your VM.

Can you try running

curl https://api.cloudquery.io/plugins

from your VM? Does it give a response?

5

Our server is from GCP Cloud and it is running in the Mumbai region. Remaining things I am checking.

6

Also, that endpoint usually responds with an HTTP redirect to storage.googleapis.com, which also needs to be allowed (if the issue is blocked outbound connections).

7

@sweeping-caribou Any luck resolving this? Let us know if we can help with anything.