Failing to connect with snowflake, when running command cloudquery sync apigee_to_snowflake.yaml

General
1

Error: failed to load spec(s) from apigee_to_snowflake.yaml. Error: failed to expand environment variable in file apigee_to_snowflake.yaml (section 2): env variable snowflake://sayedmohammad.ali@lorenipsum.com.snowflakecomputing.com/?authenticator=externalbrowser&warehouse=A202210_APIGEE_TEST_MDS_WH&db=MYDATASPACE&schema= A202210_A202210_DEMO_MDS not found

Also, I wanted to know after sync what will happen, how comes Apigee comes to know which table they need to push log data into snowflake as in above connection string we are not specifying table name. It will create table automatically into snowflake?

I followed this method Quickstart - Windows | CloudQuery

2

Hi Sayed,

Can you share the spec file (apigee_to_snowflake.yaml)? Please redact any credentials.

Also, I wanted to know after sync what will happen, how comes Apigee comes to know which table they need to push log data into snowflake as in above connection string we are not specifying table name. It will create table automatically into snowflake?

The table names are defined by the source plugin, in this case Apigee. You can see a list here.

3

Hi

In the Snowflake destination docs, the connection_string spec option has an example value of "${SNOWFLAKE_CONNECTION_STRING}". This assumes you’ve put the actual connection string inside an env variable named SNOWFLAKE_CONNECTION_STRING. To use the connection string directly in the spec, you can omit the ${...} part and refer to it as:

  spec:
    connection_string: "snowflake://sayedmohammad.ali@lorenipsum.com.snowflakecomputing.com/?authenticator=externalbrowser&warehouse=A202210_APIGEE_TEST_MDS_WH&db=MYDATASPACE&schema= A202210_A202210_DEMO_MDS"

This connection string doesn’t have a password but it would still expose the username and your Snowflake hostname this way in the file, so we still recommend you go with the environment variable method.

4

hi kemal,

Thanks for the quick response, yes, the error is gone now, I need to use esso for authentication instead of credentials as we dont have any credentials. This time it run loaded plugins then again thrown different error

Error: failed to sync v3 source apigee: failed to init destination snowflake: rpc error: code = Internal desc = failed to init plugin: failed to initialize client: authentication timed out

5

Hi Sayed,

Our Snowflake plugin does not currently support external browser authentication. You can see the supported methods here. We’ll open a feature request to track this internally.

In the meantime, the plugin is open source so we’d welcome pull requests for this feature.

6

@can thanks I tried private key but still getting error that I need to use use decrypted private key

Error: failed to sync v3 source apigee: failed to init destination snowflake: rpc error: code = Internal desc = failed to init plugin: failed to initialize client: invalid spec
private_key: encrypted private keys are not supported, use decrypted private key

7

Hi @Sayed_Mohammad_Ali is your private key a PEM-encoded file starting with “-----BEGIN PRIVATE KEY-----”?

As stated in our documentation, encrypted private keys are not supported by the upstream driver. There are instructions on that link for decrypting the key.

8

Hi @can Able to use the decrypted key but still facing issue with ssl certificate,
tls: failed to verify certificate: x509: certificate is valid for *.prod3.us-west-2.snowflakecomputing.com, *.us-west-2.snowflakecomputing.com, *.global.snowflakecomputing.com, *.snowflakecomputing.com, *.prod3.us-west-2.aws.snowflakecomputing.com,

9

Can you share the Snowflake section of your spec, omitting the credentials?

10

It was url error in connection string but not I am facing jwt token error I think I need to rotate my private key to make it work

cloudquery sync apigee_to_snowflake.yaml
Loading spec(s) from apigee_to_snowflake.yaml
time=“2025-03-25T16:21:04+05:30” level=error msg=“Authentication FAILED” func=gosnowflake.authenticate file=“auth.go:364”
Error: failed to sync v3 source apigee: failed to init destination snowflake: rpc error: code = Internal desc = failed to init plugin: failed to initialize client: 390144 (08004): JWT token is invalid. [a086908d-2e75-4018-893d-17425e6aaa82]

11

I am not able to access the schema as I am able to access it via UI
I am not able to connect with the schema.

Error: failed to sync v3 source apigee: failed to init destination snowflake: rpc error: code = Internal desc = failed to init plugin: failed to initialize client: 390201 (08004): The requested schema does not exist or not authorized.