Configuring Your Databricks Destination

The Databricks integration utilizes Unity Catalog data governance features. Be sure to enable Unity Catalog on your Databricks Workspace.

# In this article

# Step 1: Create a SQL warehouse

Create a new SQL warehouse for data writing.

1. Log in to the Databricks account.

2. In the navigation pane, click SQL Warehouses.

3. From the SQL Warehouses console, click Create SQL Warehouse.

4. From New SQL warehouse, choose a Name and configure the options for the new SQL warehouse. Under Advanced options, ensure that Unity Catalog is On, and click Create.

# Step 2: Configure Access

There are two options for configuring access to your destination warehouse:

• OAuth Secret - This is the most secure option, and is recommended for most users.

• Personal Access Token (PAT) - An option that can be used for non-interactive or automated connections, when you need to authenticate without an OAuth flow.

# (Option A) Configuring an OAuth secret

1. In your Databricks workspace, click your username or icon in the top right, click Settings, Identity and access, and next to the Service Principals options, and click Manage.

2. Click Add service principal, click Add new, enter a display name and click Add.

3. Click on the newly created Service Principal, and on the Secrets tab, click Generate secret.

4. Enter a lifetime for the secret (such as 90 days, 180 days, or 365 days), click Generate and note the Secret value and Client ID.

5. Navigate back to the SQL Warehouses section of your workspace, click the SQL Warehouses tab, and select the SQL Warehouse you created in Step 1. Click Permissions in the top right, search for and select the Service Principal you created, select Can use permission, and click Add.

6. In the Databricks UI, select the Catalog tab, and select the target Catalog. On the catalog Permissions tab, click Grant. Then, select the Service principal for which you generated the OAuth token, select USE CATALOG, and click Grant.

7. Under the target Catalog, select the target schema (main.default, or create a new target schema). On the schema Permissions tab, click Grant. Then, select the principal for which you generated the access token, and select either ALL PRIVILEGES or the following nine privileges and then click Grant:

   • USE SCHEMA
   • APPLY TAG
   • MODIFY
   • READ VOLUME
   • SELECT
   • WRITE VOLUME
   • CREATE MATERIALIZED VIEW
   • CREATE TABLE
   • CREATE VOLUME

# (Option B) Configuring a personal access token (PAT)

Collect connection information and create an access token for the data transfer service.

1. In the SQL Warehouses console, select the SQL warehouse you created in Step 1.

2. On the Connection Details tab, note the Server hostname, Port, and HTTP path for later use.

3. Click the link to create a personal access token.

4. Click Generate New Token.

5. Name the token and assign the token lifetime. A longer lifetime will help you avoid updating the token as often. Click Generate.

6. In the pop up that appears, copy and securely save the token.

(optional) If you choose to use a Service Principal & Token instead of your Personal Access Token for authentication, follow these steps:

1. In the Databricks workspace, click your username (top right), click Admin Settings > Identity and Access, and next to the Service Principals options, click Manage.

2. Click Add Service Principal, then click Add New, enter a display name, and click Add.

3. Click the newly created Service Principal, and under Entitlements, select Databricks SQL Access and Workspace Access.

4. Click Update, and note the Application ID of your newly created Service Principal.

5. In the Admin Settings menu, click Advanced (under Workspace Admin).

6. In Access Control (next to Personal Access Tokens), click Permission Settings.

7. Search for and select the Service Principal you created, select the Can Use Permission, click Add, and then Save.

8. Navigate back to the SQL Warehouses section of your workspace, click the SQL Warehouses tab, and select the SQL Warehouse you created in Step 1.

9. Click Permissions on the top right, search for and select the Service Principal you created, select the Can Use Permission, and click Add.

10. Use your terminal to generate a Service Principal Access Token using the Personal Access Token you generated above. Record the token value. This token can now be used as the access token for the connection.

    cURL request:

    curl --request POST "https://<databricks-account-id>.cloud.databricks.com/api/2.0/token-management/on-behalf-of/tokens" \
    --header "Authorization: Bearer <personal-access-token>" \
    --data '{
      "application_id": "<application-id-of-service-principal>",
      "lifetime_seconds": <token-lifetime-in-seconds-eg-31536000>,
      "comment": "<some-description-of-this-token>"
    }'
    

11. In the Databricks UI, select the Catalog tab, and select the target Catalog. Within the catalog Permissions tab, click Grant. Select the principal for whom you generated the access token, select USE CATALOG, and click Grant.

12. Under the target Catalog, select the target schema (for example, main.default), or create a new target schema. Within the schema Permissions tab, click Grant. Select the principal for whom you generated the access token, and select either ALL PRIVILEGES (or the following nine privileges), and click Grant:

    • USE SCHEMA
    • APPLY TAG
    • MODIFY
    • READ VOLUME
    • SELECT
    • WRITE VOLUME
    • CREATE MATERIALIZED VIEW
    • CREATE TABLE
    • CREATE VOLUME

# Step 3: Create a new destination

Create a new destination for data writing.

1. Log in to Iterable as a user with the Manage Integrations project permission and open the project you’re working on.

2. Go to Integrations > Data Sync.

3. Select the name of the destination that you’re connecting to, then click Continue.

4. Provide the following details:

   • Destination name – A name for the destination.
   • Server hostname – The server hostname for your Databricks workspace.
   • Port – The port for the connection.
   • Schema – The target schema name.
   • Catalog – The target catalog for your data.
   • HTTP path – The HTTP path for your destination warehouse.
   • Auth Method – Choose one of the following:
     • Client Credentials – Recommended. Enter the client ID and client secret for your OAuth application.
     • Personal Access Token (PAT) – For non-interactive or automated connections. Enter the PAT token.
   • SSH tunneling – If you require extra security, enable it and provide the necessary details.

5. Click Create Destination.

# Permissions checklist

Verify that the following permissions are applied in your Databricks project:

• Workspace: Service principal or user has permission to use the target SQL warehouse (“Can use”).

• Unity Catalog: USE CATALOG on the target catalog; appropriate privileges   on the target schema (such as, USE SCHEMA, CREATE TABLE, and read/write   volume privileges as needed), or ALL PRIVILEGES on the schema.

  If you’re using Hive Metastore (instead of UC), select hive metastore   explicitly and configure required object storage staging (bucket and   credentials), with write/delete permissions.

• Network: If IP Access Lists are enabled, the static egress IP is allowlisted   so connections and test sessions can open successfully. For US configurations, the egress IP is 35.193.179.61. For EU configurations, it is 104.155.71.71. See Databricks documentation: Databricks IP Access Lists.

# FAQ

Q: What supported authentication method is recommended to securely connect to Databricks?

A: Recommended: OAuth (client credentials) with a service principal. OAuth issues short-lived tokens, scopes access with the principal's entitlements and UC grants, and supports centralized rotation and revocation. Personal Access Tokens (PATs) are supported where policy requires, but they are long-lived bearer tokens and typically inherit broader, user-level permissions.

Q: What permissions are required?

A: The connection identity needs Can use permissions on the SQL warehouse, USE CATALOG on the target catalog, and schema-level privileges to create/manage tables (or ALL PRIVILEGES on the schema). Missing USE CATALOG is a frequent cause of test-connection failures.

Q: What credentials and connection details are required?

A: You'll need to provide the server hostname, HTTP path, Catalog, Schema, and an OAuth client (service principal) or PAT with warehouse access. Collect host/path from the SQL Warehouses console.

Q: Do you support Unity Catalog and Hive Metastore?

A: Yes. Unity Catalog is the default. For Hive Metastore, explicitly select hive in configuration and supply an S3 staging bucket and credentials. Using the wrong metastore type can cause discovery or permission issues.