How to configure cloudquery for cross account access without full org access

CloudQuery Plugins
1

Hi, how do I specify cross-account access? Not full-blown org access. I’m using CloudQuery in Account1, but I also want to grab information from Account2.

kind: source
spec:
  name: Account1
  path: cloudquery/aws
  version: "v23.1.0" 
  tables:
    - aws_s3_buckets
  skip_dependent_tables: true
  concurrency: 5000 # default 500000
  destinations: ["postgresql"]
---
kind: source
spec:
  name: Account2
  path: cloudquery/aws
  version: "v23.1.0"
  tables:
    - aws_s3_buckets
  skip_dependent_tables: true
  concurrency: 5000
  destinations: ["postgresql"]
  spec:
    accounts:
      id: Account2
      role_arn: arn:aws:iam::1234567890:role/cloudquery-read
2

That looks correct. Do you get errors?

3

Hi Michal,

If you’d like to hop on a call to discuss this at any point, here’s a link to my calendar: https://calendly.com/josh-cloudquery

4

not errors, necessarily
but it looks like the role is not used.
For account 2, I see that CQ is attempting to read resources while using the default role for account 1 (which is the role assigned to EC2 where CQ is running).
Or maybe there’s one now, with the config I submitted:

code = Internal desc = failed to init plugin: failed to initialize client: json: cannot unmarshal object into Go struct field Spec.accounts of type []spec.Account" grpc.method=Init grpc.method_type=unary grpc.service=cloudquery.plugin.v3.Plugin

But I tried with account: instead of accounts: in spec before, and this error was not present.

5

I think that you might be missing a hyphen.

Try this:

kind: source
spec:
  name: Account1
  path: cloudquery/aws
  version: "v23.1.0" 
  tables:
    - aws_s3_buckets
  skip_dependent_tables: true
  concurrency: 5000 # default 500000
  destinations: ["postgresql"]
---
kind: source
spec:
  name: Account2
  path: cloudquery/aws
  version: "v23.1.0"
  tables:
    - aws_s3_buckets
  skip_dependent_tables: true
  concurrency: 5000
  destinations: ["postgresql"]
  spec:
    accounts:
      - id: Account2
        role_arn: arn:aws:iam::1234567890:role/cloudquery-read
6

Let me check. By the way, I don’t know why there is a 3-4 minute delay between those 2 accounts. It works now, thank you @key-whippet!