# Cloudquery tables lack foreign key relationships for improved data flexibility

**URL:** <https://community.cloudquery.io/t/cloudquery-tables-lack-foreign-key-relationships-for-improved-data-flexibility/328>\
**Category:** CloudQuery Plugins\
**Created:** [March 5, 2024, 9:56am UTC](https://community.cloudquery.io/t/cloudquery-tables-lack-foreign-key-relationships-for-improved-data-flexibility/328 "2024-03-05T09:56:08Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![concise-oryx](https://avatars.discourse-cdn.com/v4/letter/c/67e7ee/32.png) [@concise-oryx](https://community.cloudquery.io/u/concise-oryx)\
**Post date:** [March 5, 2024, 9:56am UTC](https://community.cloudquery.io/t/cloudquery-tables-lack-foreign-key-relationships-for-improved-data-flexibility/328/1 "2024-03-05T09:56:08Z")

</div>

Why do the CloudQuery tables not have any foreign key relationships between them?

It might be easier for us to find the relationships between the different resource tables if we had foreign keys enabled for these tables.

---

<div class="post-metadata">

**Author:** ![erez](https://sea1.discourse-cdn.com/flex001/user_avatar/community.cloudquery.io/erez/32/40_2.png) [@erez](https://community.cloudquery.io/u/erez)\
**Post date:** [March 5, 2024, 10:39am UTC](https://community.cloudquery.io/t/cloudquery-tables-lack-foreign-key-relationships-for-improved-data-flexibility/328/2 "2024-03-05T10:39:52Z")

</div>

Hi @concise-oryx,

I commented on the issue [here](https://github.com/cloudquery/cloudquery/issues/17015).

---

<div class="post-metadata">

**Author:** ![concise-oryx](https://avatars.discourse-cdn.com/v4/letter/c/67e7ee/32.png) [@concise-oryx](https://community.cloudquery.io/u/concise-oryx)\
**Post date:** [March 5, 2024, 10:56am UTC](https://community.cloudquery.io/t/cloudquery-tables-lack-foreign-key-relationships-for-improved-data-flexibility/328/3 "2024-03-05T10:56:11Z")

</div>

```auto
failed to download plugin: failed downloading from URL [{https://github.com/cloudquery/cloudquery/releases/download/plugins-source-aws-v23.1.0/aws_windows_amd64.zip true} {https://github.com/cloudquery/cq-source-aws/releases/download/v23.1.0/cq-source-aws_windows_amd64.zip false}]. Error not found

```

I’m getting this error when trying to use the AWS version as v23.1.0.

---

<div class="post-metadata">

**Author:** ![erez](https://sea1.discourse-cdn.com/flex001/user_avatar/community.cloudquery.io/erez/32/40_2.png) [@erez](https://community.cloudquery.io/u/erez)\
**Post date:** [March 5, 2024, 10:58am UTC](https://community.cloudquery.io/t/cloudquery-tables-lack-foreign-key-relationships-for-improved-data-flexibility/328/4 "2024-03-05T10:58:20Z")

</div>

Hi @concise-oryx, can you share your CLI version and configuration content (please redact any sensitive information)?

```bash
cloudquery --version

```

---

<div class="post-metadata">

**Author:** ![concise-oryx](https://avatars.discourse-cdn.com/v4/letter/c/67e7ee/32.png) [@concise-oryx](https://community.cloudquery.io/u/concise-oryx)\
**Post date:** [March 5, 2024, 10:59am UTC](https://community.cloudquery.io/t/cloudquery-tables-lack-foreign-key-relationships-for-improved-data-flexibility/328/5 "2024-03-05T10:59:57Z")

</div>

cloudquery version 2.5.1

---

<div class="post-metadata">

**Author:** ![erez](https://sea1.discourse-cdn.com/flex001/user_avatar/community.cloudquery.io/erez/32/40_2.png) [@erez](https://community.cloudquery.io/u/erez)\
**Post date:** [March 5, 2024, 11:02am UTC](https://community.cloudquery.io/t/cloudquery-tables-lack-foreign-key-relationships-for-improved-data-flexibility/328/6 "2024-03-05T11:02:08Z")

</div>

Can you please update to the latest version? That version is quite old and doesn’t support the CloudQuery registry where new versions of plugins are hosted.  
You can use this guide to update: [CloudQuery Quickstart](https://docs.cloudquery.io/docs/quickstart)

---

<div class="post-metadata">

**Author:** ![concise-oryx](https://avatars.discourse-cdn.com/v4/letter/c/67e7ee/32.png) [@concise-oryx](https://community.cloudquery.io/u/concise-oryx)\
**Post date:** [March 5, 2024, 11:02am UTC](https://community.cloudquery.io/t/cloudquery-tables-lack-foreign-key-relationships-for-improved-data-flexibility/328/7 "2024-03-05T11:02:51Z")

</div>

Can I directly update it to the latest version??  
Or will it have any dependencies on the AWS and Azure and the Postgres versions that I am using?  
I have used this command, but still, I am getting the same version which I am currently using and unable to update it.

Command:

```auto
curl -L https://github.com/cloudquery/cloudquery/releases/download/cli-v5.8.1/cloudquery_windows_amd64.exe -o cloudquery.exe

```

---

<div class="post-metadata">

**Author:** ![erez](https://sea1.discourse-cdn.com/flex001/user_avatar/community.cloudquery.io/erez/32/40_2.png) [@erez](https://community.cloudquery.io/u/erez)\
**Post date:** [March 5, 2024, 11:09am UTC](https://community.cloudquery.io/t/cloudquery-tables-lack-foreign-key-relationships-for-improved-data-flexibility/328/8 "2024-03-05T11:09:18Z")

</div>

Can you paste the command you’re running to check the version?  
It’s best to use the latest CLI with new plugin versions, but it should be compatible with old versions too. However, the latest plugin versions do require a new CLI and won’t work with an old one.

---

<div class="post-metadata">

**Author:** ![concise-oryx](https://avatars.discourse-cdn.com/v4/letter/c/67e7ee/32.png) [@concise-oryx](https://community.cloudquery.io/u/concise-oryx)\
**Post date:** [March 5, 2024, 11:12am UTC](https://community.cloudquery.io/t/cloudquery-tables-lack-foreign-key-relationships-for-improved-data-flexibility/328/9 "2024-03-05T11:12:30Z")

</div>

Okay… makes sense checking on this… thanks for your immediate support @erez.  
Now I am able to update the CloudQuery CLI version but unable to init the AWS plugin.

```auto
Error: 
 unexpected error from sync client receive: rpc error: code = Unknown desc = failed to sync records: failed to sync unmanaged client: skip_tables include a pattern aws_redshift_endpoint_access with no matches

```

But I can see that the version of AWS Source plugin which I am using is having this table “aws\_redshift\_endpoint\_access”.

**Versions I am Using:**

- AWS: 23.1.0
- PostgreSQL: 7.0.0
- CLI: 5.8.1

---

<div class="post-metadata">

**Author:** ![erez](https://sea1.discourse-cdn.com/flex001/user_avatar/community.cloudquery.io/erez/32/40_2.png) [@erez](https://community.cloudquery.io/u/erez)\
**Post date:** [March 5, 2024, 11:49am UTC](https://community.cloudquery.io/t/cloudquery-tables-lack-foreign-key-relationships-for-improved-data-flexibility/328/10 "2024-03-05T11:49:31Z")

</div>

Hi @concise-oryx,

The table name is `aws_redshift_endpoint_accesses`. I think you’re missing `es` at the end.

---

<div class="post-metadata">

**Author:** ![concise-oryx](https://avatars.discourse-cdn.com/v4/letter/c/67e7ee/32.png) [@concise-oryx](https://community.cloudquery.io/u/concise-oryx)\
**Post date:** [March 5, 2024, 12:41pm UTC](https://community.cloudquery.io/t/cloudquery-tables-lack-foreign-key-relationships-for-improved-data-flexibility/328/11 "2024-03-05T12:41:11Z")

</div>

Hi @erez,

Now I am able to run CloudQuery for my required version, but I am getting an issue while running it.

**Error:**

```auto
unexpected error from sync client receive: rpc error: code = Unknown desc = failed to sync records: failed to sync unmanaged client: your configuration references the following premium tables: "aws_accessanalyzer_analyzer_findings_v2, aws_autoscaling_warm_pools, aws_backupgateway_gateways, aws_budgets_budgets, aws_budgets_actions, aws_cloudwatch_metrics, aws_cloudwatch_metric_statistics, aws_cod...". Please run `cloudquery login` or use a valid API Key which can be generated via https://cloud.cloudquery.io to allow the sync to succeed

```

Syncing resources… (0/-, 0 resources/hr) [0s]

---

<div class="post-metadata">

**Author:** ![erez](https://sea1.discourse-cdn.com/flex001/user_avatar/community.cloudquery.io/erez/32/40_2.png) [@erez](https://community.cloudquery.io/u/erez)\
**Post date:** [March 5, 2024, 12:41pm UTC](https://community.cloudquery.io/t/cloudquery-tables-lack-foreign-key-relationships-for-improved-data-flexibility/328/12 "2024-03-05T12:41:57Z")

</div>

Hi @concise-oryx,

As the error suggests, you’d need to run

```auto
cloudquery login

```

and authenticate to sync those tables.

See more about this in [this blog post](https://www.cloudquery.io/blog/aws-azure-gcp-pricing-changes).

---

<div class="post-metadata">

**Author:** ![concise-oryx](https://avatars.discourse-cdn.com/v4/letter/c/67e7ee/32.png) [@concise-oryx](https://community.cloudquery.io/u/concise-oryx)\
**Post date:** [March 5, 2024, 3:06pm UTC](https://community.cloudquery.io/t/cloudquery-tables-lack-foreign-key-relationships-for-improved-data-flexibility/328/13 "2024-03-05T15:06:14Z")

</div>

But they are premium tables, right?  
Do we forcefully have to be upgraded to v25.0 in CloudQuery since all tables are premium now?

Or can we use the old versions for some time?

---

<div class="post-metadata">

**Author:** ![erez](https://sea1.discourse-cdn.com/flex001/user_avatar/community.cloudquery.io/erez/32/40_2.png) [@erez](https://community.cloudquery.io/u/erez)\
**Post date:** [March 5, 2024, 3:07pm UTC](https://community.cloudquery.io/t/cloudquery-tables-lack-foreign-key-relationships-for-improved-data-flexibility/328/14 "2024-03-05T15:07:23Z")

</div>

You can still use older versions.

---

<div class="post-metadata">

**Author:** ![concise-oryx](https://avatars.discourse-cdn.com/v4/letter/c/67e7ee/32.png) [@concise-oryx](https://community.cloudquery.io/u/concise-oryx)\
**Post date:** [March 5, 2024, 3:08pm UTC](https://community.cloudquery.io/t/cloudquery-tables-lack-foreign-key-relationships-for-improved-data-flexibility/328/15 "2024-03-05T15:08:57Z")

</div>

And also, there aren’t any foreign key relationships between the tables in CloudQuery. Then how do we find the relation between the tables of different resources?

For example, suppose EC2 has `ELbv2LoadBalancers`, `ElbV2Listeners`, and `Ec2Eips`, etc. How can we find the relation between these tables?

There might be some use cases where we need the Load Balancers and Listener information of EC2 instances. How do we find the relation between them?

Thanks, makes sense.

---

<div class="post-metadata">

**Author:** ![erez](https://sea1.discourse-cdn.com/flex001/user_avatar/community.cloudquery.io/erez/32/40_2.png) [@erez](https://community.cloudquery.io/u/erez)\
**Post date:** [March 5, 2024, 3:13pm UTC](https://community.cloudquery.io/t/cloudquery-tables-lack-foreign-key-relationships-for-improved-data-flexibility/328/16 "2024-03-05T15:13:27Z")

</div>

Each table has a `_cq_id` and `_cq_parent_id` columns, so you should be able to use those to link between children and parents.

You can get all the relations via the `cloudquery tables` command, as explained in [this GitHub issue](https://github.com/cloudquery/cloudquery/issues/17015#issuecomment-1978459713).

Can you specify the table names you’d like to correlate? For example, `aws_elbv2_listeners` is a child of `aws_elbv2_load_balancers`, so `aws_elbv2_listeners` will have `_cq_parent_id` pointing to `_cq_id` of `aws_elbv2_load_balancers`.

---

<div class="post-metadata">

**Author:** ![concise-oryx](https://avatars.discourse-cdn.com/v4/letter/c/67e7ee/32.png) [@concise-oryx](https://community.cloudquery.io/u/concise-oryx)\
**Post date:** [March 5, 2024, 3:35pm UTC](https://community.cloudquery.io/t/cloudquery-tables-lack-foreign-key-relationships-for-improved-data-flexibility/328/17 "2024-03-05T15:35:43Z")

</div>

So you are saying that for the child table, we have `_cq_parent_id`, and for the parent table, we have `_cq_id`. So can we perform a join directly, or should we create a foreign key relationship between them?

And also, if I need to join `aws_elbv2_target_group_target_health_descriptions` and `aws_elbv2_listeners` tables, how do I achieve this? Because both of these tables have different parents.

---

<div class="post-metadata">

**Author:** ![erez](https://sea1.discourse-cdn.com/flex001/user_avatar/community.cloudquery.io/erez/32/40_2.png) [@erez](https://community.cloudquery.io/u/erez)\
**Post date:** [March 5, 2024, 3:42pm UTC](https://community.cloudquery.io/t/cloudquery-tables-lack-foreign-key-relationships-for-improved-data-flexibility/328/18 "2024-03-05T15:42:05Z")

</div>

I think you should do a join. I would not create a foreign key, as syncing data from multiple APIs is not an atomic operation, so data can get stale while we sync it or even have transient errors.

What are you trying to achieve with the join between `aws_elbv2_listeners` and `aws_elbv2_target_group_target_health_descriptions`?

---

<div class="post-metadata">

**Author:** ![concise-oryx](https://avatars.discourse-cdn.com/v4/letter/c/67e7ee/32.png) [@concise-oryx](https://community.cloudquery.io/u/concise-oryx)\
**Post date:** [March 5, 2024, 3:43pm UTC](https://community.cloudquery.io/t/cloudquery-tables-lack-foreign-key-relationships-for-improved-data-flexibility/328/19 "2024-03-05T15:43:16Z")

</div>

Sorry, I didn’t get this. You mean I should create new foreign keys for them?  
We can get the listeners that are not attached to the target group like that?

---

<div class="post-metadata">

**Author:** ![erez](https://sea1.discourse-cdn.com/flex001/user_avatar/community.cloudquery.io/erez/32/40_2.png) [@erez](https://community.cloudquery.io/u/erez)\
**Post date:** [March 5, 2024, 3:45pm UTC](https://community.cloudquery.io/t/cloudquery-tables-lack-foreign-key-relationships-for-improved-data-flexibility/328/20 "2024-03-05T15:45:19Z")

</div>

I would recommend not creating foreign keys as that might make the schema too hard to modify/extend.

Looks like `aws_elbv2_target_groups` has a `load_balancer_arns` column.

[Next page](https://community.cloudquery.io/t/cloudquery-tables-lack-foreign-key-relationships-for-improved-data-flexibility/328.md?page=2)
