Configuring Connection Details

This section provides instructions for configuring the connection details required to establish communication between your PubSub+ event broker and your third-party system.

Solace PubSub+ Connection Details

The Spring Cloud Stream Binder for PubSub+ uses Spring Boot Auto-Configuration for the Solace Java API to configure its session. In the application.yml, this typically is configured as follows:

solace:
  java:
    host: tcp://localhost:55555
    msg-vpn: default
    client-username: default
    client-password: default

For more information and options to configure the PubSub+ session, see Spring Boot Auto-Configuration for the Solace Java API.

Preventing Message Loss when Publishing to Topic-to-Queue Mappings

If the connector is publishing to a topic that is subscribed to by a queue, messages may be lost if they are rejected (for example, if queue ingress is shut down).

To prevent message loss, configure the reject-msg-to-sender-on-discard option with the including-when-shutdown flag.

Microsoft SQL Server Connection Details

To manually configure the Microsoft SQL Server connection details, set the following properties in application.yml:

spring:
  cloud:
    stream:
      sqlservercdc:
        bindings:
          input-0:
            consumer:
              endpoint:
                query-parameters:
                  databaseHostname: <hostname>
                  databasePort: <port>
                  databaseUser: <username>
                  databasePassword: <password>
                  databaseNames: <dbname>
                  ...

Where:

  • databaseHostname is the hostname of the Microsoft SQL Server database server.

  • databasePort is the port of the Microsoft SQL Server database server.

  • databaseNames is the name of the Microsoft SQL Server database.

  • databaseUser is the user to authenticate with Microsoft SQL Server database.

  • databasePassword is the password to use for authentication.

Additional connection parameters are available; some important ones are listed in the following table. For details about all the available properties, see the documentation for the Debezium SQL Server Component of Apache Camel.

Parameter Type Description
databaseInstance String The hostname of the Microsoft SQL Server instance.
topicPrefix String

A logical namespace for the Microsoft SQL Server database server or cluster. The topicPrefix is an internal identifier that helps distinguish between different Microsoft SQL Server sources when multiple connectors are running. It must contain only alphanumeric characters, hyphens, dots, and underscores. If not configured, it defaults to the binding name.

The topicPrefix does not become an SMF topic.
snapshotMode String

The criteria for running a snapshot upon startup of the connector. The allowed values are:

  • initial—The default. Takes a snapshot on first startup, then streams changes.

  • always —Performs snapshot on every connector start.

  • initial_only—Takes a snapshot but doesn't transition to streaming.

  • no_data—Schema structure only, no data.

  • never—Never takes snapshots.

  • recovery —Takes snapshot when no offsets are available.

  • when_needed—Takes a snapshot when the connector determines it's needed.

  • configuration_based—Uses connector-specific configuration.

  • custom—Uses a custom snapshotter implementation.
skippedOperations String

A comma-separated list of operations to skip during streaming. The allowed values are: 

  • c—Skip create/insert operations.

  • u—Skip update operations.

  • d—Skip delete operations.

  • t—Skip truncate operations (default).

  • none—Skip nothing (process all operations).

For example, skippedOperations=t,d indicates to skip truncations and deletions.
tombstonesOnDelete boolean Indicates whether delete operations should be represented by a delete event and a subsequent tombstone event. When tombstonesOnDelete=true, a delete operation generates two events: a delete event and a subsequent tombstone event (key with null value). This allows message brokers with log compaction to completely remove all events for a deleted row's key.
timePrecisionMode String

Times, dates, and timestamps can be represented with different kinds of precision, including:

  • adaptive—The default. Captures time and timestamp values exactly as in the database using millisecond, microsecond, or nanosecond precision values based on the database column's type.
  • adaptive_time_microseconds—Always uses microsecond precision for time fields.
  • connect—Always represents time and timestamp values using millisecond precision regardless of the database column's precision.
decimalHandlingMode String

Specifies how decimal and numeric columns should be represented in change events. Valid options include:

  • precise—Represents numbers precisely in binary form using java.math.BigDecimal values.
  • double—Represents numbers using Java double values, which may result in a loss of precision but are easier to use.
  • string—Encodes numbers as formatted strings. Semantic information about the real type is lost.

Checkpoint Store Configuration

The Connector for Microsoft SQL Server CDC stores the information about the current progress of processing files in a checkpoint store backed by a Last Value Queue (LVQ) on a PubSub+ event broker. The checkpoint store contains information about the files that have been processed and the files that are currently being processed.

The following table lists the configuration options for the Checkpoint Store.

Parameter Type Default Description
solace.sqlservercdc.checkpoint.lvqName String - Required. The name of the Solace LVQ to be used for checkpointing.
solace.sqlservercdc.checkpoint.autoProvisionLvq boolean false Indicates whether to auto-provision the LVQ queue if it does not exist.

Microsoft SQL Server CDC Binder Configuration Options

The following properties are available at the Microsoft SQL Server CDC binder level. These properties are to be prefixed with spring.cloud.stream.cdc-sqlserver.bindings.<inputname>.consumer. for source bindings and spring.cloud.stream.cdc-sqlserver.bindings.<outputname>.producer. for target bindings.

Microsoft SQL Server CDC Consumer Configuration Options

The following configuration options are available for Microsoft SQL Server CDC consumers.

Standard Spring Cloud Stream Properties

Config Option Type Valid Values Default Value Description
spring.cloud.stream.bindings.<input-x>.destination String - - The source table from which to consume CDC events, in the format: schema.tableName. For example, public.products.
spring.cloud.stream.bindings.<input-x>.binder String - - Must be set to sqlservercdc.

Custom Properties

Config Option Type Valid Values Default Value Description
payloadSchemaEnabled boolean true/false false Indicates whether to include the schema information in the final payload.
skipDeleteEvents boolean true/false true Indicates whether delete events will be skipped.

Headers

The connector injects the following CDC-related headers in each message:

Header Name Type Description
cdc_operation String Values: c (create/insert), u (update), d (delete), r (read)
cdc_db_name String The name of the database
cdc_schema String The name of the schema
cdc_table_name String The name of the table
cdc_commit_lsn String Commit log sequence number
cdc_change_lsn String Change log sequence number
cdc_key String The key of the event
cdc_before String State of the row before the event occurred
cdc_timestamp Long The time at which the connector processed the event

Connecting to Multiple Systems

To connect to multiple systems of a same type, use the multiple binder syntax.

For example:

spring: 
  cloud:
    stream: 
      binders:
        
        # 1st solace binder in this example 
     solace1:
          type: solace 
       environment: 
         solace: 
           java:
             host: tcp://localhost:55555
         
     # 2nd solace binder in this example 
     solace2:
          type: solace 
          environment: 
            solace: 
           java:
             host: tcp://other-host:55555
        
     # The only cdc-sqlserver binder 
     cdc-sqlserver1:
          type: cdc-sqlserver
       # Add `environment` property map here if you need to customize this binder. 
       # But for this example, we'll assume that defaults are used.
          
     # Required for internal use 
     undefined:
          type: undefined 
     bindings:
          input-0:
            destination: <input-destination> 
         binder: cdc-sqlserver1
          output-0:
            destination: <output-destination>
            binder: solace1 # Reference 1st solace binder 
       input-1:
            destination: <input-destination> 
         binder: cdc-sqlserver1
          output-1:
            destination: <output-destination>
            binder: solace2 # Reference 2nd solace binder

The configuration above defines two binders of type solace and one binder of type cdc-sqlserver, which are then referenced within the bindings.

Each binder above is configured independently under spring.cloud.stream.binders.<bindername>.environment..

  • When connecting to multiple systems, all binder configuration must be specified using the multiple binder syntax for all binders. For example, under the spring.cloud.stream.binders.<binder-name>.environment.

  • Do not use single-binder configuration (for example, solace.java.* at the root of your application.yml) while using the multiple binder syntax.