How to set different tenant_id for source and destination in CloudQuery

Hello team,

I am trying to retrieve data from Azure in one tenant and write it into a blob storage in another tenant. I have created a multiple tenant application so I can use the same app for both source and destination. However, the tenant_id needs to be different between the source and the destination.

Any idea how I can set a different tenant_id between the source and destination? In the documentation, I can specify only one environment variable (AZURE_TENANT_ID), which will be used for both source and destination. Is there a way to specify in the configuration a different one for the destination?

Thank you for your help :slightly_smiling_face:

Hi @obliging-kangaroo :wave:

This reminded me of an old issue that was similar: #3551 on GitHub. At the time, it was closed as won’t fix.

However, it seems like in that request, the problem was using multiple tenants on the source side, whereas you’re asking about different tenants between source and destination.

What you’re asking is technically possible via a workaround, but it’s a little bit tricky/annoying :slightly_smiling_face:

What I think you can do is run the destination plugin as a standalone gRPC server, then update your config to connect to it via gRPC. If you do this, you’d have the ability to provide a different TENANT_ID to the destination than to the source.

If this doesn’t make sense let me know, I’d be happy to expand a bit more.

Actually, let me just say one more thing.

To download the destination plugin and run it as a gRPC server you can:

  1. Run cloudquery install, then
  2. Run something like:
    .cq/plugins/destination/cloudquery/azblob/v{VERSION}/plugin serve
    

It will then print the address, usually 127.0.0.1:7777, which you’ll need to set as the path in your destination config. You should also change the registry value to grpc.

The thing that makes it annoying, though, is that you’ll then need to stop the destination plugin server yourself, because the CloudQuery CLI won’t have ownership over the process.

Hello @herman, thank you for your quick reply! I think we are going to fix it by copying the files into a local directory in the server and then do an az copy to our blob storage. That seems to me as the easiest solution, without adding too much complexity.