Changelog discrepancy for CloudQuery AWS source version updates

CloudQuery Plugins
1

Hi Team,

I see v23.2.0 got published today for AWS source. But I fail to read what has been changed. The Changelog.md is still referring to 22.19.2 as the latest release.

What is the major change from 22 to 23? I’m now on 22.15.0 and want to upgrade to the latest version. Is there a special page where I can find this?

EDIT: My bad - I was still used to the changelog in GitHub. I see a changelog has been added on the website as a separate tab as well :slightly_smiling_face:

Still leaves my question open: why is the GitHub CHANGELOG.md still on 22.19.2?
GitHub CHANGELOG.md

2

Hi @correct-boa,

You can see the changelog here.

Ah, I just saw your edit :slightly_smiling_face:. Since the AWS plugin is open core (part free, part paid), the GitHub open source changelog reflects the last version of the 100% free version. The source of truth for plugins should be the Hub going forward.

More in this blog post.

3

Thanks for the fast answer @erez. What is, in general, the recommended procedure for breaking changes on the tables we’re using?

Are the scripts automatically altering the table schemas and we only have to alter our queries?

E.g. I see a lot of primary keys have been added to tables we’re using. Does this require a deletion from the table on our end?
(using destination PostgreSQL)

4

Hi @correct-boa, you’re correct v23.0.0 had quite a few breaking changes to the schema that we had backed up. CloudQuery has automatic schema migration to some extent which is described in:

CloudQuery Migrations

For breaking changes which cannot be migrated automatically without data loss, you can use migrate_mode: forced on the destination, and CloudQuery will drop the relevant tables, losing any data. See:

CloudQuery Destination Spec - Migrate Mode

You might also be interested in two other configuration options (going forward): pk_mode and deterministic_cq_id:

CloudQuery Destination Spec - PK Mode

CloudQuery Source Spec - Deterministic CQ ID

If you use both of those together, breaking changes to PKs should not require dropping tables.

5

Hmm, but where can I find the v23?
Simply bumping the version in my config.yaml seems not to work. (I’m not performing a cloudquery login)

Reading the blog post on Open Core states I should not have to change anything to be able to use v23 if I only want to keep using the free tables?

6

@correct-boa Adding registry: cloudquery (or updating the CLI to one of the newer versions and removing registry: github) should work.

7

@correct-boa You should be able to use version: v23.0.0 and registry: cloudquery without login unless you reference paid tables.
You’ll get a relevant error message if you do.