Error: failed to load spec(s) from apigee_to_snowflake.yaml. Error: failed to expand environment variable in file apigee_to_snowflake.yaml (section 2): env variable snowflake://sayedmohammad.ali@lorenipsum.com.snowflakecomputing.com/?authenticator=externalbrowser&warehouse=A202210_APIGEE_TEST_MDS_WH&db=MYDATASPACE&schema= A202210_A202210_DEMO_MDS not found
Also, I wanted to know after sync what will happen, how comes Apigee comes to know which table they need to push log data into snowflake as in above connection string we are not specifying table name. It will create table automatically into snowflake?
Can you share the spec file (apigee_to_snowflake.yaml)? Please redact any credentials.
Also, I wanted to know after sync what will happen, how comes Apigee comes to know which table they need to push log data into snowflake as in above connection string we are not specifying table name. It will create table automatically into snowflake?
The table names are defined by the source plugin, in this case Apigee. You can see a list here.
In the Snowflake destination docs, the connection_string spec option has an example value of "${SNOWFLAKE_CONNECTION_STRING}". This assumes you’ve put the actual connection string inside an env variable named SNOWFLAKE_CONNECTION_STRING. To use the connection string directly in the spec, you can omit the ${...} part and refer to it as:
This connection string doesn’t have a password but it would still expose the username and your Snowflake hostname this way in the file, so we still recommend you go with the environment variable method.
Thanks for the quick response, yes, the error is gone now, I need to use esso for authentication instead of credentials as we dont have any credentials. This time it run loaded plugins then again thrown different error
Error: failed to sync v3 source apigee: failed to init destination snowflake: rpc error: code = Internal desc = failed to init plugin: failed to initialize client: authentication timed out
Our Snowflake plugin does not currently support external browser authentication. You can see the supported methods here. We’ll open a feature request to track this internally.
In the meantime, the plugin is open source so we’d welcome pull requests for this feature.