Hi team,
I developed a Stripe to S3 integration by using SQLite to save the state, and I am using the free version of this integration. However, it keeps writing the same 1 row of Stripe charge every day to S3 and doesn’t update the state. Can you please help me fix the below YAML files?
s3.yaml:
kind: destination
spec:
name: "s3"
path: "cloudquery/s3"
version: "v4.10.0"
write_mode: "append"
spec:
bucket: "xxx-xxx-datalake"
region: "eu-west-2"
path: "raw_data/stripe_raw_data/{{TABLE}}/dt={{YEAR}}-{{MONTH}}-{{DAY}}/{{UUID}}.parquet"
format: "parquet"
athena: true
stripe.yaml:
kind: source
spec:
name: "stripe"
path: "cloudquery/stripe"
version: "v2.1.11"
registry: github
tables:
- stripe_charges
destinations:
- s3
spec:
api_key: ${STRIPE_TOKEN}
rate_limit: 20
incremental_sync: true
backend_options:
table_name: "cq_state_stripe"
connection: "@@plugins.sqlite.connection"
sqlite.yaml:
kind: destination
spec:
name: "sqlite"
path: "cloudquery/sqlite"
version: "v2.4.13"
spec:
connection_string: /cloudquery/state.sqlite
Thank you!