CloudQuery Azure configuration for ECS ClientID and Password setup

CloudQuery Plugins
1

:wave: I feel like I’m asking a very simple question, but I’m getting a bit stuck.
I’ve been using CloudQuery for 1 year on AWS, and I use ECS in which I have a configuration file that includes credentials.
I don’t understand how to do this with Azure?
I have a Client ID and its Password, but then? How and where to specify them so that they are used from ECS?
Thanks :slightly_smiling_face:

2

Hi @adjusted-lioness,

You can see the authentication docs for Azure here. The Azure plugin also supports all authentication methods supported by the Azure Go SDK. This is yet to be fully documented (see this issue), but you can look at the Azure SDK for Go documentation.

I opened a PR to update the docs: PR #15646.

3

@adjusted-lioness Ideally, you should avoid embedding the credentials in configuration files, and use either environment variables or a secret management service, like for example Secrets Manager in AWS.

4

Thanks a lot @erez. I’ve seen the documentation, but I’m not really comfortable with the environment variable.

May I ask for an example for Azure? To use with:

AZURE_TENANT_ID
AZURE_CLIENT_ID
AZURE_CLIENT_SECRET

Do I have to use something like:

spec:
  service_account_key_json: |
    ${GCP_SERVICE_ACCOUNT_KEY_JSON}

I’ve tried to add these keys as Docker environment variables, but it appeared that this was not the right solution. Indeed, and thanks for this reminder. Once I understand it for Azure, I will apply it for AWS.

But as far as I understand, it’s still recommended for AWS to specify the role in the configuration, isn’t it? Do you recommend a better secure approach?

Multi-Account AWS Documentation

5

Hi @adjusted-lioness, you don’t need to add anything to the YAML file spec. You only need those environment variables to be available in the shell running the CLI + Plugin. You can export environment variables in Unix-like machines via:

export AZURE_TENANT_ID=<YOUR AZURE_TENANT_ID>
export AZURE_CLIENT_ID=<YOUR AZURE_CLIENT_ID>
export AZURE_CLIENT_SECRET=<YOUR AZURE_CLIENT_SECRET>
6

We use Fargate ECS. How can I use the export variables? As the “machine” will be deleted after each execution. Locally, I do not have any interrogations about the how-to; it’s really on such a use case with a temporary instance.

7

Hi @adjusted-lioness,

If the question is how to pass environment variables to ECS, there are some docs on it here: Specifying Sensitive Data, specifically Using Secrets as Environment Variables.

8

Hi @erez,

I retrieved the error message that was:

missing environment variable AZURE_CLIENT_ID

I am trying to retrieve the ECS configuration we’re using, since we’re utilizing the environment variables as described in the AWS documentation. I had a flash when I saw that among all the errors, only the client_id was missing. When I checked the configuration, there was a space that had slipped into the variable.

I’m really sorry; I was quite embarrassed after reading and re-reading the documentation, trying things out, and seeing that despite everything, I was still failing.

Once again, thank you for your help, and for CloudQuery, quite simply!

9

Hi @adjusted-lioness, no worries! I’m glad it’s working now.