Cloud Data Ingestion for Google BigQuery

This page covers everything you need to sync data from Google BigQuery to Braze using Cloud Data Ingestion (CDI): the resources to create in Google Cloud, how to connect the source and create a sync in Braze, the source table for each data type, and BigQuery-specific troubleshooting.

Setup overview

1. In your BigQuery account, set up the tables or views you want to sync to Braze.
2. Create a service account and allow access to the BigQuery projects and datasets that contain the data you want to sync.
3. Set up how Braze accesses the service account, using one of the supported authentication methods.
4. Create a new source and sync in the Braze dashboard.
5. Validate the integration and start the sync.

Step 1: Set up your source table

The steps in this section create a table with a PAYLOAD column. If you plan to map columns with Visual or write a query with SQL on the Data definition step, you can use an existing table or view instead. Skip the table creation step, and grant Braze read access to the tables or views you want to sync. For a comparison of options, see Choose a data definition option.

You can name the project, dataset, and table as you’d like, but the column names should match the definitions in the following tabs. You may also use a view or a materialized view instead of a table.

Optionally, set up a new project or dataset to hold your source table:

1

CREATE SCHEMA BRAZE-CLOUD-PRODUCTION.INGESTION;

note

If you use a PAYLOAD column, your source table or view can include columns that aren’t listed in the following tabs (for example, auditing or hashing). Braze reads only the columns described in these tabs; other columns are not used during Cloud Data Ingestion syncs.

Select the tab for the data type you want to sync.

• user attributes
• custom events
• purchase events
• catalog items
• canvas triggers
• user deletions

User attributes, custom events, and purchase events all use the same source table shape. Only the contents of PAYLOAD differ.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

CREATE TABLE `BRAZE-CLOUD-PRODUCTION.INGESTION.USERS_ATTRIBUTES_SYNC`
(
  updated_at TIMESTAMP DEFAULT current_timestamp,
  --At least one of external_id, alias_name and alias_label, or braze_id is required
  external_id STRING,
  --If using user alias, both alias_name and alias_label are required
  alias_name STRING,
  alias_label STRING,
  --braze_id can only be used to update existing users created through the Braze SDK
  braze_id STRING,
  --If you include both email and phone, email is used as the primary identifier
  email STRING,
  phone STRING,
  payload JSON
);

Field Name	Type	Mode
UPDATED_AT	TIMESTAMP	REQUIRED
PAYLOAD	JSON	REQUIRED
EXTERNAL_ID	STRING	NULLABLE
ALIAS_NAME	STRING	NULLABLE
ALIAS_LABEL	STRING	NULLABLE
BRAZE_ID	STRING	NULLABLE
EMAIL	STRING	NULLABLE
PHONE	STRING	NULLABLE

• UPDATED_AT - The time this row was updated in or added to the table. Braze syncs rows where UPDATED_AT is later than the last synced value. Rows at the exact boundary timestamp may be re-synced if new rows share that same timestamp.
• User identifier columns - Your table may contain one or more user identifier columns. Each row should only contain one identifier (either external_id, the combination of alias_name and alias_label, braze_id, email or phone). A source table may have columns for one, two, three, four, or all five identifier types.
  • EXTERNAL_ID - This identifies the user you want to update. This should match the external_id value used in Braze.
  • ALIAS_NAME and ALIAS_LABEL - These two columns create a user alias object. alias_name should be a unique identifier, and alias_label specifies the type of alias. Users may have multiple aliases with different labels but only one alias_name per alias_label.
  • BRAZE_ID - The Braze user identifier. This is generated by the Braze SDK, and new users cannot be created using a Braze ID through Cloud Data Ingestion. To create new users, specify an external user ID or user alias.
  • EMAIL - The user’s email address. If multiple profiles with the same email address exist, the most recently updated profile is prioritized for updates. If you include both email and phone, email is used as the primary identifier.
  • PHONE - The user’s phone number. If multiple profiles with the same phone number exist, the most recently updated profile is prioritized for updates.
• PAYLOAD - This is a JSON string of the fields you want to sync to the user in Braze.

For attribute syncs, PAYLOAD follows the user attributes object format. You can include nested custom attributes, arrays of objects, and subscription group statuses. For payload examples, see Table setup.

important

BigQuery partitioning

CDI supports partitions for BigQuery. If you partition by a function of UPDATED_AT (for example, at the granularity of a day, week, or hour, depending on the size of your dataset), BigQuery can prune the data it needs to scan. This improves performance and efficiency for very large tables.

Don’t partition by any other fields. Test different configurations to find the best setup for your specific data.

All CDI queries filter by UPDATED_AT, but this behavior could change. Design your table schema to not require that queries include this clause.

For more information, refer to the BigQuery partitioning documentation.

Custom events use the same source table as user attributes. Create the table shown in the User attributes tab, then populate PAYLOAD with an events object.

To sync events, an event name is required. Format the time field as an ISO 8601 string or in yyyy-MM-dd'T'HH:mm:ss:SSSZ format. If the time field is not present, Braze uses the UPDATED_AT column value as the event time. Other fields, including app_id and properties, are optional.

You can sync one event per row.

1
2
3
4
5
6
7
8
9

{
    "app_id" : "your-app-id",
    "name" : "rented_movie",
    "time" : "2013-07-16T19:20:45+01:00",
    "properties": {
        "movie": "The Sad Egg",
        "director": "Alex Smith"
    }
}

Purchase events use the same source table as user attributes. Create the table shown in the User attributes tab, then populate PAYLOAD with a purchases object.

To sync purchase events, product_id, currency, and price are required. Format the optional time field as an ISO 8601 string or in yyyy-MM-dd'T'HH:mm:ss:SSSZ format. If the time field is not present, Braze uses the UPDATED_AT column value as the event time. Other fields, including app_id, quantity, and properties, are optional.

You can sync one purchase event per row.

1
2
3
4
5
6
7
8
9
10
11
12

{
    "app_id" : "11ae5b4b-2445-4440-a04f-bf537764c9ad",
    "product_id" : "Completed Order",
    "currency" : "USD",
    "price" : 219.98,
    "time" : "2013-07-16T19:20:30+01:00",
    "properties" : {
        "products" : [ { "name": "Monitor", "category": "Gaming", "product_amount": 19.99 },
        { "name": "Gaming Keyboard", "category": "Gaming ", "product_amount": 199.99 }
        ]
    }
}

Before you create a catalog sync, create the target catalog in Braze. For the full catalog workflow, see Sync and delete catalog data.

1
2
3
4
5
6
7

CREATE TABLE `BRAZE-CLOUD-PRODUCTION.INGESTION.CATALOGS_SYNC`
(
  updated_at TIMESTAMP DEFAULT current_timestamp,
  id STRING,
  payload JSON,
  deleted BOOLEAN
);

Field Name	Type	Mode
UPDATED_AT	TIMESTAMP	REQUIRED
PAYLOAD	JSON	REQUIRED
ID	STRING	REQUIRED
DELETED	BOOLEAN	OPTIONAL

The name of the integration must exactly match the name of the catalog you created in Braze.

Braze recommends creating a view from your catalog data so the source table fully refreshes each time a sync runs. For example, if you have a table of product data (product_catalog_1) with product_id and three additional attributes, you could sync the following view:

1
2
3
4
5
6
7
8
9
10
11

CREATE view IF NOT EXISTS BRAZE_CLOUD_PRODUCTION.INGESTION.CATALOGS_SYNC AS (SELECT
    last_updated as UPDATED_AT,
    product_id as ID,
    TO_JSON(
      STRUCT(
      attribute_1,
      attribute_2,
      attribute_3,
      )
    ) as PAYLOAD
  FROM `BRAZE_CLOUD_PRODUCTION.INGESTION.product_catalog_1`);

Canvas triggers power zero-copy personalization. For the destination Canvas setup and considerations, see Zero-copy personalization.

1
2
3
4
5
6
7
8
9
10

CREATE TABLE `BRAZE-CLOUD-PRODUCTION.INGESTION.CANVAS_TRIGGERS_SYNC`
(
  updated_at TIMESTAMP DEFAULT current_timestamp,
  --At least one of external_id or alias_name and alias_label is required
  external_id STRING,
  --If using user alias, both alias_name and alias_label are required
  alias_name STRING,
  alias_label STRING,
  properties JSON
);

Field Name	Type	Required?
UPDATED_AT	TIMESTAMP	Yes
PROPERTIES	JSON	Yes
EXTERNAL_ID	STRING	NULLABLE
ALIAS_NAME	STRING	NULLABLE
ALIAS_LABEL	STRING	NULLABLE

Canvas trigger syncs differ from other user data syncs in three ways:

• Only external ID or user alias identifiers are supported. Email and phone numbers are not supported identifiers.
• Only existing Braze users can be synced. New users cannot be created.
• PROPERTIES replaces the PAYLOAD column. This is a JSON string of the fields you want to use as Canvas entry properties for personalization.

Your Canvas triggers table needs the following columns:

• UPDATED_AT - The time this row was updated or added to the table. Braze syncs rows where UPDATED_AT is later than the last synced value. Rows at the exact boundary timestamp may be re-synced if new rows share that same timestamp.
• Either EXTERNAL_ID or ALIAS_NAME and ALIAS_LABEL as the user identifier column. These identify the users for whom you want to trigger Canvas messaging.
  • EXTERNAL_ID - Identifies the user to enter into the Canvas. This should match the external_id value used in Braze.
  • ALIAS_NAME and ALIAS_LABEL - These columns create a user alias object. alias_name should be a unique identifier, and alias_label specifies the alias type. Users may have multiple aliases with different labels, but only one alias_name per alias_label.
• PROPERTIES - A JSON string of fields to make available as personalization properties in your Canvas. This should contain user-specific information.

note

Properties are not required for every row or user. However, properties values must be a valid JSON string. Input an empty {} string if there are no properties for the row.

To get started, select the Canvas Triggers data type when creating a new sync.

Source tables for user deletes should include one or more user identifier types and an UPDATED_AT timestamp. Payload columns aren’t supported for user delete data.

• UPDATED_AT - The time this row was updated or added to the table. Braze syncs rows where UPDATED_AT is later than the last synced value. Rows at the exact boundary timestamp may be re-synced if new rows share that same timestamp.
• User identifier columns - Each row should only contain one identifier: either external_id, the combination of alias_name and alias_label, or braze_id. A source table may contain columns for one, two, or all three identifier types.
  • EXTERNAL_ID - This identifies the user you want to update. This should match the external_id value used in Braze.
  • ALIAS_NAME and ALIAS_LABEL - These two columns create a user alias object. alias_name should be a unique identifier, and alias_label specifies the type of alias. Users may have multiple aliases with different labels but only one alias_name per alias_label.
  • BRAZE_ID - The Braze user identifier. This is generated by the Braze SDK and new users cannot be created using a Braze ID through Cloud Data Ingestion. To create new users, specify an external user ID or user alias.

important

Do not include a PAYLOAD column in your table for user removal. To prevent accidental, permanent removal of users, a sync fails if a payload column is provided in the source table. Any other columns are allowed but are ignored by Braze.

Create a table with the following fields:

Field Name	Type	Mode
UPDATED_AT	TIMESTAMP	REQUIRED
EXTERNAL_ID	STRING	NULLABLE
ALIAS_NAME	STRING	NULLABLE
ALIAS_LABEL	STRING	NULLABLE
BRAZE_ID	STRING	NULLABLE

On the Create import sync page, set the Data Type to Delete Users so that the proper actions are taken during the integration run. For more information, see Delete users with Cloud Data Ingestion.

warning

Deleting user profiles cannot be undone. The delete action permanently removes users, which may cause discrepancies in your data. For details, see Effects of deleting user profiles.

Step 2: Create a service account and grant permissions

Create a service account in GCP for Braze to use to connect and read data from your table(s). The service account should have the following permissions:

• BigQuery Connection User: Allows Braze to make connections
• BigQuery User: Provides Braze access to run queries, read dataset metadata, and list tables.
• BigQuery Data Viewer: Provides Braze access to view datasets and their contents.
• BigQuery Job User: Provides Braze access to run jobs

Step 3: Set up authentication

Braze can access the service account with any of the following authentication methods:

Workload Identity Federation

With Workload Identity Federation (WIF), Braze authenticates to Google Cloud with a short-lived identity from its own AWS environment, and then impersonates the service account you created for Braze. You configure your project to trust that identity for your workspace only. You don’t create or share a service account key.

You set up WIF once per Google Cloud project. One WIF credential serves both BigQuery and Google Cloud Storage sources. If you’ve already completed these steps for a Google Cloud Storage source, grant the existing service account the permissions for this source and skip the rest of this section.

Prerequisites

To complete these steps in the Google Cloud console, you need permission to create workload identity pools and to manage Identity and Access Management (IAM) policies on the service account and your data. For example, you can use the Workload Identity Pool Admin (roles/iam.workloadIdentityPoolAdmin) and Service Account Admin (roles/iam.serviceAccountAdmin) roles.

Trust only your workspace’s principal

warning

Grant access to the full Braze principal Amazon Resource Name (ARN) from your credential form. Don’t grant access to the Braze AWS role or AWS account. A role-level or account-level grant lets other Braze customers read your data.

Every Braze workspace on the same Braze instance presents the same AWS role. Only the session name at the end of the principal ARN, which is your workspace ID, is unique to you.

Google’s default attribute mapping for AWS providers sets aws_role to the role ARN without the session name. This means granting access to identities that match an aws_role value, such as principalSet://iam.googleapis.com/projects/PROJECT_NUMBER/locations/global/workloadIdentityPools/POOL_ID/attribute.aws_role/ROLE_ARN, trusts every Braze workspace on your instance, including yours.

To keep your data scoped to your workspace:

• Grant access only to identities whose subject is your full Braze principal ARN. This is the principal://iam.googleapis.com/projects/PROJECT_NUMBER/locations/global/workloadIdentityPools/POOL_ID/subject/BRAZE_PRINCIPAL_ARN principal.
• Don’t grant access to All identities in the pool, or to identities filtered by aws_role or account.
• Don’t write an attribute condition that checks only the AWS account or role.
• Keep the default attribute mapping for google.subject, which is assertion.arn.

The Braze principal ARN differs for each workspace and Braze instance, and Braze uses a separate AWS role for Google sources than for other sources such as Snowflake. Always copy the value from a BigQuery or Google Cloud Storage credential form in the workspace you’re connecting.

Copy the Braze principal ARN

In Braze, start creating your BigQuery source and select Workload Identity Federation as the authentication method. Copy the value in step 1 of the credential form, Copy the Braze principal ARN, and keep the form open.

If the field shows Unavailable, WIF isn’t available for your Braze instance. Use a service account key instead.

Enable the required APIs

In the Google Cloud console, go to APIs & Services > Library, and enable the following APIs for your project:

• IAM Service Account Credentials API
• Security Token Service API
• Identity and Access Management (IAM) API

Create the workload identity pool and provider

1. Go to IAM & Admin > Workload Identity Federation, and then select Create pool.
2. Enter a name for the pool, such as “Braze CDI”. Note the Pool ID, and then select Continue.
3. For Select a provider, select AWS.
4. Enter a provider name, such as “Braze AWS”, and note the Provider ID.
5. For AWS account ID, enter the 12-digit number that follows arn:aws:sts:: in the Braze principal ARN, and then select Continue.
6. Under Configure provider attributes, keep the default mapping, including google.subject = assertion.arn.
7. Select Save.

Pool and provider IDs must be 4 to 32 characters, using lowercase letters, numbers, and hyphens.

Restrict the provider to your principal (optional)

For defense in depth, add an attribute condition so the provider accepts only your workspace’s principal. Google then rejects every other identity at the token exchange, before Google checks any service account permission.

1. In the pool, select the provider, and then select Edit.
2. Under Attribute conditions, select Add condition.

3. Enter the following condition, replacing BRAZE_PRINCIPAL_ARN with the full value that you copied from the credential form:

   
   1
   
    assertion.arn == 'BRAZE_PRINCIPAL_ARN'
   
   

4. Select Save.

This condition dedicates the provider to one Braze workspace. To connect another workspace, create a separate provider for it, or list both principals, as in assertion.arn in ['FIRST_BRAZE_PRINCIPAL_ARN', 'SECOND_BRAZE_PRINCIPAL_ARN'].

Bind the Braze principal to the service account

Grant your workspace’s principal permission to impersonate the service account:

1. Go to IAM & Admin > Workload Identity Federation, and then select your pool.
2. Select Grant access, and then select Grant access using service account impersonation.
3. For Service account, select the service account you created for Braze.
4. Under Select principals, select Only identities matching the filter.
5. For Attribute name, select subject. For Attribute value, paste the full Braze principal ARN from the credential form.
6. Select Save. If Google offers to download a client library configuration file, select Dismiss. Braze doesn’t need it.

This grants the Workload Identity User role (roles/iam.workloadIdentityUser) on the service account to a principal in this form:

1

principal://iam.googleapis.com/projects/PROJECT_NUMBER/locations/global/workloadIdentityPools/POOL_ID/subject/BRAZE_PRINCIPAL_ARN

To confirm, open the service account and select the Permissions tab. The Workload Identity User role should list this principal, ending in your full Braze principal ARN.

warning

Don’t select All identities in the pool, and don’t filter by aws_role. Both match every Braze workspace on your Braze instance, because they all share the same AWS role. For details, see Trust only your workspace’s principal.

Grant this role on the service account only. A project-level grant lets the principal impersonate every service account in the project.

IAM changes can take a few minutes to take effect. If a connection test fails right after you grant access, wait a few minutes and try again.

The service account also needs access to your data. Make sure you’ve granted it the permissions for this source. If you use one credential for both BigQuery and Google Cloud Storage, grant both sets of permissions to the same service account.

Enter the values in Braze

Return to the Braze credential form, and enter the following values from the Google Cloud console:

Braze combines the project number, pool ID, and provider ID into the audience, which tells Google which provider validates the request from Braze. By default, a provider accepts its own resource name as the audience. If you set custom allowed audiences on your provider, include the provider’s full resource name.

How Braze tests the connection

When you test the connection in Braze, Braze also checks that your configuration trusts only your workspace. During the test, Braze attempts the same exchange with a test session name of all zeros, in this form:

1

assumed-role/cdi-gcs-bigquery-sync-<deployment>/000000000000000000000000

Your configuration must reject this identity, either at the token exchange or when it tries to impersonate your service account. If both steps succeed, the test fails, because your configuration trusts identities other than your workspace’s principal. To fix this, refer to Your configuration trusts identities other than your workspace.

If you set the optional attribute condition, Google rejects the test identity at the token exchange. Otherwise, Google rejects the test identity when that identity tries to impersonate your service account. Either way, the test passes.

You may see this rejected attempt in your Google Cloud audit logs each time you test the connection. This behavior is expected. The attempt comes from the Braze check, and the attempt doesn’t give anyone access to your data.

Don’t add a condition that rejects only this test session. Such a condition passes the test while your configuration still trusts other Braze workspaces.

Remove Braze access

To stop Braze from accessing your data, remove the Workload Identity User role for the Braze principal from the service account’s Permissions tab, or delete the provider.

Set up with the gcloud CLI

If you prefer the command line, the following commands enable the APIs, create the pool and provider, and bind the Braze principal to your service account. Replace the values in the first block, and paste the Braze principal ARN exactly as copied.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

PROJECT_ID="YOUR-PROJECT-ID"
POOL_ID="braze-cdi"
PROVIDER_ID="braze-aws"
SA_EMAIL="YOUR-SERVICE-ACCOUNT-EMAIL"
BRAZE_PRINCIPAL_ARN="PASTE-THE-BRAZE-PRINCIPAL-ARN-FROM-YOUR-CREDENTIAL-FORM"

PROJECT_NUMBER="$(gcloud projects describe "$PROJECT_ID" --format='value(projectNumber)')"
BRAZE_AWS_ACCOUNT_ID="$(echo "$BRAZE_PRINCIPAL_ARN" | cut -d: -f5)"

gcloud services enable iam.googleapis.com iamcredentials.googleapis.com sts.googleapis.com \
  --project="$PROJECT_ID"

gcloud iam workload-identity-pools create "$POOL_ID" \
  --project="$PROJECT_ID" --location="global" --display-name="Braze CDI"

gcloud iam workload-identity-pools providers create-aws "$PROVIDER_ID" \
  --project="$PROJECT_ID" --location="global" --workload-identity-pool="$POOL_ID" \
  --account-id="$BRAZE_AWS_ACCOUNT_ID"

# Optional: restrict the provider to your principal
gcloud iam workload-identity-pools providers update-aws "$PROVIDER_ID" \
  --project="$PROJECT_ID" --location="global" --workload-identity-pool="$POOL_ID" \
  --attribute-condition="assertion.arn == '${BRAZE_PRINCIPAL_ARN}'"

gcloud iam service-accounts add-iam-policy-binding "$SA_EMAIL" \
  --project="$PROJECT_ID" \
  --role="roles/iam.workloadIdentityUser" \
  --member="principal://iam.googleapis.com/projects/${PROJECT_NUMBER}/locations/global/workloadIdentityPools/${POOL_ID}/subject/${BRAZE_PRINCIPAL_ARN}"

Service account key

Braze authenticates with a JSON key for your service account. Existing service account key credentials keep working, and you don’t need to migrate them.

In the Google Cloud console, open the service account, go to Keys > Add key > Create new key, and select JSON. Download the key file. You upload it to Braze when you create the source.

Alternatively, use gcloud:

1
2

gcloud iam service-accounts keys create braze-cdi-key.json \
  --iam-account=SERVICE_ACCOUNT_EMAIL

Replace SERVICE_ACCOUNT_EMAIL with the email address of the service account. For more information, see Create and delete service account keys.

Step 4: Allow access to Braze IPs

If you have network policies in place, you must give Braze network access to your BigQuery instance. Allow access from the following IPs corresponding to your Braze dashboard’s region.

• united states (us)
• european union (eu)
• australia (au)
• indonesia (id)
• japan (jp)
• south korea (kr)

Step 5: Create a new source in the Braze dashboard

In the Braze dashboard, go to Data Settings > Cloud Data Ingestion > Sources, select Add data source, and then select Google BigQuery.

Step 5.1: Add BigQuery connection information and source table

Enter a name for your source, and then enter your BigQuery credentials and configuration:

• For Credential name, enter a label that helps you identify these credentials in Braze.
• For the authentication method, select the method you set up in Step 3, and enter its credentials. For Workload Identity Federation, enter Project number, Workload identity pool ID, Provider ID, and Service account email. For Service account key, enter JSON key file for the service account.

Then, input the remaining configuration fields.

Step 5.2: Test connection and connect to source

Next, select Test connection. Once successful, finalize remaining settings and click Connect to Source. If the connection fails, an error message appears to help troubleshoot the issue.

Step 6: Create a new sync in the Braze dashboard

Go to Data Settings > Cloud Data Ingestion > Syncs, and select Create data sync.

Configure sync details and define your data

Enter a name for your sync. Then, select an active source and a sync data type. Next, select Visual or Table and enter your source table, or select SQL to write a custom SQL query. Finally, select Preview and validate.

After validation succeeds, a preview of the data appears. Select Next: Notifications to continue. If validation fails, an error message appears to help troubleshoot the issue.

note

You must successfully validate a sync before progressing to next steps. If you need to close out of the sync creation page, select Save as draft to keep your work in progress.

Add notification preferences

Input contact email(s) for sync error notifications. Braze uses this contact information to send notifications about integration errors, such as unexpected loss of table access.

Contact emails only receive notifications of global or sync-level errors such as missing tables, permissions, and others. They do not receive row-level issues. Global errors indicate critical problems with the connection that prevent syncs from running.

Such problems can include the following:

• Connectivity issues
• Lack of resources
• Permissions issues
• Catalog tier is out of space (catalog syncs only)

Set the schedule

Lastly, configure your sync as non-recurring or recurring.

Non-recurring syncs run only when you select Sync Now in the dashboard or call the Trigger a sync endpoint.

Recurring syncs can run as often as every 5 minutes or as rarely as once per month. By default, the shortest interval you can set in the dashboard is 15 minutes, which helps manage your warehouse compute costs and request volume. To sync as often as every 5 minutes, contact Braze Support or your customer success manager. Braze schedules recurring syncs in UTC.

note

You must successfully validate an integration before it can move from Draft to Active state. If you close the creation page, your integration is saved, and you can revisit the details page to make changes and validate.

Run the sync

When activated, your sync runs on the schedule configured during setup. If you want to run the sync outside its schedule or to fetch the most recent data, select Sync Now. This run does not impact regularly scheduled future syncs. To start a sync automatically when your data updates, call the Trigger a sync endpoint from your pipeline.

Set up additional integrations or users (optional)

You can set up multiple integrations with Braze. When creating additional syncs, you can reuse existing credentials if connecting to the same BigQuery account.

If you reuse the same user across integrations, you cannot delete the user in the Braze dashboard until it’s removed from all active syncs.

Use BigQuery as a connected source

A connected source is a zero-copy alternative to syncing data into Braze. Braze queries your BigQuery instance directly to build CDI segments, without copying the underlying data. For concepts, limitations, and cost implications, see Connected sources.

Step 1: Create a service account and grant permissions

Create a service account in GCP for Braze to use to connect and read data from your table(s). The service account should have the following permissions:

• BigQuery Connection User: Allows Braze to make connections.
• BigQuery User: Provides Braze access to run queries, read dataset metadata, and list tables.
• BigQuery Data Viewer: Provides Braze access to view datasets and their contents.
• BigQuery Job User: Provides Braze access to run jobs.
• bigquery.tables.create: Provides Braze access to create temporary tables during segment refresh.

This service account is used to generate the list of tables available in your CDI Segment Extensions, and to query source tables to create new segments. After the connected source is created, Braze discovers the names and description of all tables available to the user in the source schema.

You may choose to grant access to all tables in a dataset, or grant privileges only to specific tables. Whichever tables the Braze role has access to are available to query in the CDI Segment Extension.

The create table permission is required so Braze can create a table with your CDI Segment Extension query results before updating the segment in Braze. Braze creates a temporary table per segment, and the table only persists while Braze is updating the segment.

After creating the service account and granting permissions, set up how Braze accesses it with one of the methods in Step 3: Set up authentication. If you already set up Workload Identity Federation for this project, you can reuse it. Grant the existing service account the permissions in this step.

Network access setup matches Step 4.

Step 2: Create the connected source in Braze

Go to Data Settings > Cloud Data Ingestion > Connected Sources, select Add data source, and then select Google BigQuery.

In Setup source, enter the following:

• Credentials: Credential name and the fields for your authentication method. For Workload Identity Federation, enter Project number, Workload identity pool ID, Provider ID, and Service account email. For Service account key, enter JSON key file for the service account.

• Configuration: Project and Dataset

Choose a name for the connected source. This name is used in the list of available sources when you create a new CDI Segment Extension.

Configure a maximum runtime for this source. Braze automatically aborts any queries that exceed the maximum runtime. The maximum runtime allowed is 60 minutes; a lower runtime reduces costs incurred on your BigQuery account. This setting applies to queries run through this source, including syncs and CDI Segment Extensions that use it.

Select Test Connection to verify that the list of tables visible to the user is what you expect, then select Done. Your connected source is now created and ready to use in CDI Segment Extensions.

You may set up multiple connected sources with Braze, but each source should be configured to connect a different dataset. When creating additional sources, you may reuse existing credentials if connecting to the same BigQuery account.

Format a payload from existing columns

If you prefer to store each attribute in its own column internally, you need to convert those columns to a JSON string to populate the sync with Braze. Use this query in BigQuery to format source columns into CDI fields.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

CREATE OR REPLACE TABLE BRAZE.EXAMPLE_USER_DATA (attribute_1 string,
     attribute_2 STRING,
     attribute_3 NUMERIC,
     my_user_id STRING);

SELECT
    CURRENT_TIMESTAMP as UPDATED_AT,
    my_user_id as EXTERNAL_ID,
    TO_JSON(
      STRUCT(
        'attribute_1' AS attribute_1,
        'attribute_2'AS attribute_2,
        'yet_another_attribute'AS attribute_3
      )
    ) as PAYLOAD
  FROM BRAZE.EXAMPLE_USER_DATA;

For more optimization guidance, see Cloud Data Ingestion best practices.

Troubleshooting

Test Connection runs slow

Test Connection is running on your data warehouse, so increasing warehouse capacity may improve its speed. Using a serverless SQL instance minimizes warmup time and improves query throughput, but may result in slightly higher integration costs.

User does not have permission to query table

If you receive this error, add user permissions to query the table.

Your usage exceeded the custom quota

If you receive this error, your quota needs to be updated so you can continue syncing at your current rate.

Table was not found in location {region} Location

If you receive this error, verify your table is in the correct project and dataset.

Invalid JWT Signature

If you receive this error, check that the BigQuery API service is enabled for your account.

Your configuration trusts identities other than your workspace

If you use Workload Identity Federation, the connection test can fail with the following error:

Your workload identity pool trusts every Braze customer, not only this workspace. Braze authenticated against it with a test identity (000000000000000000000000) that belongs to no Braze workspace, and your Google Cloud project accepted it, so another Braze customer could read this data. Grant the Workload Identity User role on your service account to the full Braze principal shown on the credential screen (principal://…/subject/<full ARN>) rather than to the role (principalSet://…/attribute.aws_role/…); restricting the pool provider’s attribute condition to that same principal works as well. Then test the connection again.

This means Google accepted the Braze test identity at both the token exchange and the service account impersonation, as described in How Braze tests the connection. This error usually happens when the service account grants access to all identities in the pool, or to identities filtered by aws_role.

To fix this error:

1. In the Google Cloud console, open the service account and select the Permissions tab.
2. Find every principal with the Workload Identity User role that starts with principalSet:// and references your pool. Remove the role from each one.
3. Make sure your full principal has the role, as described in Bind the Braze principal to the service account.
4. Go to IAM & Admin > IAM and check for project-level Workload Identity User grants to your pool. Remove them.
5. Test the connection again.

Alternatively, restrict the provider’s attribute condition to your full principal, as described in Restrict the provider to your principal (optional). Google then rejects every other identity at the token exchange. We still recommend granting access to your full principal only.

Braze couldn’t confirm that your pool trusts only this workspace

If you use Workload Identity Federation, the connection test can fail with the following error:

Braze could not confirm that your workload identity pool trusts only this workspace. The check needs Google to explicitly refuse an identity that is not yours, and this attempt failed for another reason instead. Please try again; if it keeps happening, contact support.

The Braze check described in How Braze tests the connection didn’t reach a result. For example, a network error or a temporary Google Cloud error interrupted the check. This error doesn’t mean your configuration is too broad. Test the connection again. If the error keeps happening, contact Braze Support.

Permission denied when Braze impersonates the service account

If you use Workload Identity Federation and the connection test fails with a permission error on iam.serviceAccounts.getAccessToken, check the following:

• The service account’s Permissions tab lists the Workload Identity User role for a principal that ends in your full Braze principal ARN, copied from a BigQuery or Google Cloud Storage credential form in the same workspace.
• The IAM Service Account Credentials API is enabled.
• You’ve waited a few minutes since granting access.

The token exchange is rejected

If you use Workload Identity Federation and the connection test fails at the Google Security Token Service token exchange, check the following:

• The project number, pool ID, and provider ID that you entered in Braze match your Google Cloud configuration.
• The provider’s AWS account ID matches the account ID in the Braze principal ARN.
• If you set an attribute condition, it contains your full principal ARN, exactly as copied.
• If you set custom allowed audiences, they include the provider’s full resource name.
• The pool and provider are enabled.

Where to run SQL checks

Run validation checks directly in the BigQuery Studio SQL workspace, against the same table or view used by your CDI integration. For more information, see BigQuery Studio introduction.

Related pages

• Table setup for required columns and payload formatting
• Best practices for sync scheduling and data point efficiency
• Sync logs and observability for monitoring sync runs
• FAQ for common questions across all sources