Databricks Delta Sharing

Databricks Delta Sharing allows you to securely share live Braze engagement and campaign data into your Databricks environment. This article describes how sharing works from Braze as the data provider to your Databricks account as the recipient, and how to query shared tables.

important

Databricks Delta Sharing with Braze is only available for customers hosted on our US clusters. Support for additional regions is in progress. If you are interested in Delta Sharing but are not hosted on our US cluster, contact your customer success manager.

Databricks Delta Sharing is part of Braze Data Distribution. For a full overview of Data Distribution options, see Data Distribution.

Prerequisites

Requirement	Description
Availability	Customers hosted on our US clusters. If you are interested in Delta Sharing but are not hosted on our US cluster, contact your customer success manager.
Braze workspace permissions	View Currents Integrations to view Data Sharing. Edit Currents Integrations to create, update, or delete a Delta share.

For Databricks, data sharing happens between a data provider and a data recipient. Your Braze account is the data provider because it creates and sends the share, and your Databricks account is the data recipient because it consumes the share to create a catalog you can query. For more details, see Databricks documentation on reading data shared using Databricks-to-Databricks Delta Sharing (for recipients).

In Braze, go to Partner Integrations > Data Sharing > Databricks Delta Sharing.
Enter your Databricks sharing identifier.
When you’re finished, select Create Datashare. Braze sends the share to your Databricks account.

Step 2: Create a catalog in Databricks

After a few minutes, you should receive the inbound share in your Databricks account.

Using the inbound share, create a catalog to view and query the tables. For example:

 CREATE CATALOG [IF NOT EXISTS] <catalog-name> USING SHARE braze.<share-name>;

Grant privileges so the right users and groups can query the new catalog.

warning

Shared data is read-only in your Databricks workspace. You can query it like other data, but you can’t modify or delete rows in the shared tables through the share.

Usage and visualization

After the data share is provisioned, create a catalog from the incoming share so shared tables appear in your Databricks workspace and are queryable like other data you store there. The shared data remains read-only.

Similar to Currents, you can use Databricks Delta Sharing to:

Create complex reports
Perform attribution modeling
Secure sharing within your own company
Map raw event or user data to a CRM (like Salesforce)
And more

For a full list of tables and columns available in Databricks, download the Databricks raw table schemas as a text file. This file reflects the Databricks Delta Sharing schema (for example, DB_CREATED_AT for ingestion time). It is not interchangeable with the Snowflake raw table schemas or the SQL table reference, which describe Snowflake naming and fields.

note

Column names and types can differ from Snowflake Data Sharing (for example, DB_CREATED_AT instead of SF_CREATED_AT).

User ID schema

Note the following differences between Braze and Databricks naming conventions for user IDs.

Braze schema	Databricks schema	Description
`braze_id`	`USER_ID`	The unique identifier that Braze assigns automatically.
`external_id`	`EXTERNAL_USER_ID`	The unique identifier of a user’s profile that you set in Braze.

Important information and limitations

Availability

Databricks Delta Sharing is only available for customers hosted on our US clusters. Support for additional regions is in progress. If you are interested in Delta Sharing but are not hosted on our US cluster, contact your customer success manager.

Breaking versus non-breaking changes

Non-breaking changes

Non-breaking changes can happen at any time and generally provide additional functionality. Examples of non-breaking changes:

Adding a new table or view
Adding a column to an existing table or view

important

Because new columns are considered non-breaking, Braze strongly recommends explicitly listing the columns of interest in each query instead of using SELECT * queries. Alternatively, create views that explicitly name columns and query those views instead of querying the shared tables directly.

Breaking changes

When possible, breaking changes are preceded by an announcement and a migration period. Examples of breaking changes include:

Removing a table or view
Removing a column from an existing table or view
Changing the type or nullability of an existing column

Retention policy

You can query against the most recent two years of data for each event in the corresponding USERS_*_SHARED view.

Nearly every event record Braze stores includes a few fields representing users’ personally identifiable information (PII). Some events may include email address, phone number, device ID, language, gender, and location information. If a user’s request to be forgotten is submitted to Braze, we will null out those PII fields for any event belonging to those users. This way, we’re not removing the historical record of the event, but now the event can never be tied back to a specific individual.

Querying shared data: `TIME` and query performance

Event data in the data sharing views (for example, USERS_BEHAVIORS_CUSTOMEVENT_SHARED) is clustered on the TIME field. When you filter by when the event occurred, use TIME as the preferred filter. Queries that restrict rows using TIME are generally more performant than queries that filter on DB_CREATED_AT, because clustering aligns with event time.

Field	Meaning
`TIME`	Unix timestamp at which the event happened. Prefer this when filtering by occurrence time.
`DB_CREATED_AT`	Timestamp when the row was loaded into Databricks (ingestion time).

Speed, performance, and cost of queries

The speed, performance, and cost of any query you run on top of the data depend on the SQL warehouse size you use. Depending on how much data you access, you may need a larger warehouse for the query to complete successfully. For more information, see Databricks documentation on creating and configuring a SQL warehouse (including cluster size and scaling).

New Stuff!