# CloudQuery + Databricks Integration: Complete Setup Guide

**URL:** <https://community.cloudquery.io/t/cloudquery-databricks-integration-complete-setup-guide/1206>\
**Category:** Announcements\
**Created:** [July 29, 2025, 9:13pm UTC](https://community.cloudquery.io/t/cloudquery-databricks-integration-complete-setup-guide/1206 "2025-07-29T21:13:12Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![JoeKarlsson](https://sea1.discourse-cdn.com/flex001/user_avatar/community.cloudquery.io/joekarlsson/32/37_2.png) [@JoeKarlsson](https://community.cloudquery.io/u/JoeKarlsson)\
**Post date:** [July 29, 2025, 9:13pm UTC](https://community.cloudquery.io/t/cloudquery-databricks-integration-complete-setup-guide/1206/1 "2025-07-29T21:13:12Z")

</div>

Hey everyone,

We’ve been working on running CloudQuery syncs directly _within_ Databricks, and after hitting a few walls, we’ve got a solid setup that’s working well in production. Figured I’d share the complete walkthrough since several folks have asked about this integration.

Our team needed centralized cloud asset inventory across AWS accounts, but didn’t want to spin up separate infrastructure just for CloudQuery. Since we’re already heavy Databricks users, running everything as Jobs made sense.

**The setup breakdown:**

**Secrets management** (this part’s crucial): We’re using Databricks CLI to create a secrets scope. You’ll need these keys:

```auto
AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY / AWS_SESSION_TOKEN
CLOUDQUERY_API_KEY  
DATABRICKS_CATALOG / DATABRICKS_SCHEMA
DATABRICKS_ACCESS_TOKEN / DATABRICKS_HOSTNAME / DATABRICKS_HTTP_PATH
DATABRICKS_STAGING_PATH

```

 ![image](https://us1.discourse-cdn.com/flex001/uploads/cloudquery1/original/1X/5fd88f13510088aff03dd60bc0c2da0ec51cd9d2.jpeg)

You can checkout the complete walk though here: [How to Work with CloudQuery Syncs within Databricks | CloudQuery Blog](https://www.cloudquery.io/blog/how-to-work-with-cloudquery-syncs-within-databricks)

**Additional Resources:**

- Main sync job: [https://github.com/cloudquery/databricks-simple-cli-job](https://github.com/cloudquery/databricks-simple-cli-job)

- Transformation code: [https://github.com/cloudquery/databricks-sync-transformations](https://github.com/cloudquery/databricks-sync-transformations) (private repo, ping us if you need access)

- Visualization app: [https://github.com/cloudquery/databricks-sample-app](https://github.com/cloudquery/databricks-sample-app)

---

<div class="post-metadata">

**Author:** ![stefan](https://sea1.discourse-cdn.com/flex001/user_avatar/community.cloudquery.io/stefan/32/75_2.png) [@stefan](https://community.cloudquery.io/u/stefan)\
**Post date:** [July 30, 2025, 7:19am UTC](https://community.cloudquery.io/t/cloudquery-databricks-integration-complete-setup-guide/1206/2 "2025-07-30T07:19:56Z")

</div>