Configuring Connection Details

This section provides instructions for configuring the connection details required to establish communication between the Micro-Integration and your third-party system.

For information about configuring the connection to the event broker, see Connecting to Your Event Broker .

For information about configuring error handling, see Error Handling.

Before you configure connection details, ensure you meet the prerequisites. For more information, see Before You Begin.

The name of the binder for Databricks Zerobus is dbzerobus.

spring:
  cloud:
    stream:
      <binder-name>:
        default:
          <shared parameters>
        bindings:
          input-0:
            <workflow-specific parameters>

Setting Up a Service Principal

The Micro-Integration for Databricks Zerobus uses OAuth 2.0 service principal credentials to authenticate with Databricks Zerobus.

To obtain OAuth 2.0 credentials for the Micro-Integration, create a Databricks-managed service principal. Do not use an Entra ID or Azure AD service principal.

1. In the Databricks workspace, go to Settings > Identity and access > Service principals > Manage.

2. Click + Add service principal > Add new, enter a name for the service principal, then click Add.

3. On the Secrets (or Credentials) tab, click Generate secret.

   The Client Secret is shown only once. Copy and save it immediately to a secure location. It cannot be retrieved after you close the dialog.

4. On the Configurations tab, copy the Application ID (UUID). This is your Client ID.

5. Grant the service principal the minimum required privileges on the target Unity Catalog table. Run the following SQL in a Databricks notebook or SQL editor, replacing the placeholders with your catalog, schema, table, and service principal UUID:

   GRANT USE CATALOG ON CATALOG <catalog>                        TO `<service-principal-UUID>`;
   GRANT USE SCHEMA  ON SCHEMA  <catalog>.<schema>               TO `<service-principal-UUID>`;
   GRANT MODIFY, SELECT ON TABLE <catalog>.<schema>.<table>      TO `<service-principal-UUID>`;

   MODIFY and SELECT must both be granted explicitly, even if ALL PRIVILEGES was previously granted at a higher level.

For more information, see:

• Manage service principals and Authorize service principal access with OAuth in the AWS Databricks documentation

• Manage service principals and Authorize service principal access to Azure Databricks with OAuth in the Azure Databricks documentation

Finding Your Connection Values

To find your Workspace URL and Server Endpoint, open your Databricks workspace in a browser. The full URL has this format:

https://<workspace-domain>/?o=<workspace-id>

• Workspace URL: The base URL before /?o=. For example:

  • AWS: https://<id>.cloud.databricks.com

  • Azure: https://adb-<workspace-id>.<suffix>.azuredatabricks.net

• Workspace ID: The numeric value after o= in the full browser URL.

• Server Endpoint: Constructed from the workspace ID and region. For example:

  • AWS: https://<workspace-id>.zerobus.<region>.cloud.databricks.com

  • Azure: https://<workspace-id>.zerobus.<azure-region>.azuredatabricks.net

Add the following connection settings to your application.yml file:

spring:
  cloud:
    stream:
      dbzerobus:
        default:
          databricks:
            url: https://<workspace-id>.cloud.databricks.com
            serverEndpoint: https://<workspace-id>.zerobus.<region>.cloud.databricks.com
            clientId: <client-id>
            clientSecret: <client-secret>

Databricks Zerobus Connection Details

The following table describes the Databricks Zerobus connection properties.

Databricks Zerobus Binder Configuration Options

The following properties are available at the Databricks Zerobus binder level. These properties are to be prefixed with spring.cloud.stream.dbzerobus.bindings.<outputname>.producer. for target bindings.

Databricks Zerobus Producer Configuration Options

The following configuration options are available for the Databricks Zerobus producers.

The Micro-Integration requires a fully-qualified table name (FQTN) in the format catalog.schema.table. You can provide the FQTN directly as the workflow destination, or use any combination of the workflow destination and the producer properties below to compose it. For example, if the workflow destination is set to schema.table, set catalog here to supply the missing segment.

Connecting to Multiple Systems

To connect to multiple systems of the same type, use the multiple binder syntax.

For example:

spring:
  cloud:
    stream:
      binders:

        # 1st solace binder in this example
        solace1:
          type: solace
          environment:
            solace:
              java:
                host: tcp://localhost:55555

        # 2nd solace binder in this example
        solace2:
          type: solace
          environment:
            solace:
              java:
                host: tcp://other-host:55555

        # The only databricks-zerobus binder
        databricks-zerobus1:
          type: databricks-zerobus
          # Add `environment` property map here if you need to customize this binder.
          # But for this example, we'll assume that defaults are used.

        # Required for internal use
        undefined:
          type: undefined
      bindings:
        input-0:
          destination: <input-destination>
          binder: solace1 # Reference 1st solace binder
        output-0:
          destination: <output-destination>
          binder: databricks-zerobus1
        input-1:
          destination: <input-destination>
          binder: solace2 # Reference 2nd solace binder
        output-1:
          destination: <output-destination>
          binder: databricks-zerobus1

The configuration above defines two binders of type solace and one binder of type databricks-zerobus, which are then referenced within the bindings.

Each binder above is configured independently under spring.cloud.stream.binders.<bindername>.environment..