Need help with AWS plugin deprecation notifications in CloudQuery

CloudQuery Plugins
1

Hi Team,

Currently, I am utilizing the AWS and Postgres plugins to pull my AWS cloud data. It is working fine, but we have observed that the AWS plugin has been deprecated; hence, our scheduled sync failed.

Do we have any RSS feeder or some way to know the plugin deprecation notifications?

2

Hi Phani :wave:

I’m not sure what’s happening; could you share the error you’re seeing, and the associated CloudQuery config you’re using (with sensitive information redacted)?

We do our best not to break existing deployments when releasing new versions, so I suspect there might be a configuration problem at play here.

3

Actually, we are using this image in our deployments.

image:
  repository: ghcr.io/cloudquery/cloudquery
  tag: latest

Whenever a Docker container is created, it is pulling the latest image, and hence it is breaking with old versions. I am thinking? Please correct me if I am wrong here.

Currently, my CloudQuery config is using AWS v22.6.0 and it is not visible on our release pages.

I got the following error:

Error: failed to download plugin source cloudquery/aws@v22.6.0: plugin version not found. If you're trying to use a private plugin you'll need to run `cloudquery login` first. Hint: make sure to use the latest plugin version from hub.cloudquery.io or to keep using an outdated version add `registry: github` to your configuration
2024-02-04T02:00:06Z ERR exiting with error error="failed to download plugin source cloudquery/aws@v22.6.0: plugin version not found. If you're trying to use a private plugin you'll need to run `cloudquery login` first. Hint: make sure to use the latest plugin version from hub.cloudquery.io or to keep using an outdated version add `registry: github` to your configuration" module=cli
4

The latest tag there is referring to the latest version of the CLI. We generally keep it backwards-compatible, but sometimes we need to make a breaking change, indicated with a major version bump. If you want your deployment to continue working smoothly without configuration changes, you should use a specific tag for pulling the image, instead of latest.

As the error message suggests, you can fix this specific error by adding registry: github to your source and destination configurations (docs: CloudQuery Documentation). This will let you continue using the older versions of the plugins.

In the latest major CLI version (v5), we changed the default registry to be cloudquery. When you next update to a new version of the AWS / Postgres plugins, you should switch to this registry, and then you can use any of the versions listed on our Hub here: CloudQuery Hub.

5

Thanks, Herman, for the valuable suggestions.

How can I know which AWS and PostgreSQL plugins to use for specific Docker versions of CloudQuery?

6

They should all work; we keep the CLI backward-compatible. In this case, because you upgraded from v4 to v5 of the CLI, it just requires a small configuration change to keep loading the plugin from the GitHub registry.