Microsoft SQL Server Bulk CDC

The Micro-Integration for Microsoft SQL Server Bulk CDC captures changes (inserts, updates, deletes) from multiple Microsoft SQL Server database tables simultaneously (using Debezium) and streams them to topics on an event broker service. Only header mapping is supported; payload mapping is not available for this Micro-Integration.

You can configure Microsoft SQL Server only as a source.

To configure the connection details for Microsoft SQL Server in your Micro-Integration, see Microsoft SQL Server Bulk CDC Connection Parameters.

You must also define one Micro-Integration Flow that specifies:

  • the endpoints where data is consumed from and published to. For more information, see Micro-Integration Flow Parameters.

  • how the header fields should be mapped. For more information, see Mapping Headers and Payloads.

    For bulk CDC Micro-Integrations, payload transformations are not supported. Header transformations and Smart Topics are supported.

For message headers, see Microsoft SQL Server Bulk CDC Message Headers .

Microsoft SQL Server Bulk CDC Connection Parameters

The following table describes the connection parameters for Microsoft SQL Server.

Field Description
Database Hostname The hostname of the Microsoft SQL Server database server.
Database Port The port of the Microsoft SQL Server database server.
Database Instance The name of the Microsoft SQL Server instance.
Database Name The name of the Microsoft SQL Server database.
Skip Delete Events Select to skip the processing of delete events from the database change stream.
Authentication

The authentication scheme to use when connecting to Microsoft SQL Server. Select one of:

Additional Properties

(Optional) Any additional configuration related to Microsoft SQL Server that applies to all Flows. Click Add Context if you need to specify additional context properties. For more information, see Additional Properties.
Checkpoint Queue The name of the Last Value Queue (LVQ) on the Solace event broker to use for checkpointing. The LVQ stores information about the change events that have been processed and those that are currently being processed. The LVQ must exist on the same event broker and VPN as the target destination.
Auto Create Checkpoint Queue Select to automatically create the LVQ if it does not exist.

Basic Authentication Parameters

Field Description
Username The username to use for logging in to Microsoft SQL Server.
Password The password to use for authenticating with the Microsoft SQL Server database.

SSL Certificate Authentication (SQL Server) Parameters

Field Description
Trust Server Certificate (Optional) Set to true to accept the server's certificate without validating it. Use this only if the certificate is self-signed or its hostname does not match the server address. Leave it empty to validate the certificate against the provided truststore.
SSL Truststore The truststore file (.jks or .p12) that contains the server's certificate. This is used to verify the identity of the Microsoft SQL Server server.
Truststore Type The type of the SSL truststore, either JKS or PKCS12.
Truststore Password (Optional) The password for the SSL truststore.
Username The username to use for logging in to Microsoft SQL Server.
Password The password to use for authenticating with the Microsoft SQL Server database.

Micro-Integration Flow Parameters

You must configure the endpoint parameters for the Flow. The Flow shares the connection details of the parent Micro-Integration.

The following table describes the parameters for configuring Microsoft SQL Server as a source.

Field Description
Destination

The tables from which to consume change events. Each entry must include the schema name and the table name in <schemaName>.<tableName> format. You can specify multiple tables using a semicolon-separated list, a regular expression, or a combination of both. For example:

  • Semicolon-separated list: dbo.customers;dbo.orders

  • Regular expression: dbo.table_.*

  • Combination: dbo.orders_.*;dbo.customers_.*

Additional Properties

(Optional) Any additional configuration related to Microsoft SQL Server that is specific to this Flow. Click Add Context if you need to specify additional context properties. For more information, see Additional Properties.

Additional Properties

The following table lists additional properties that are available.

Field Description
topicPrefix

A logical namespace for the Microsoft SQL Server database server or cluster. The topicPrefix is an internal identifier that helps distinguish between different Microsoft SQL Server sources when multiple Micro-Integrations are running. It must contain only alphanumeric characters, hyphens, dots, and underscores. If not configured, it defaults to the binding name.

The topicPrefix does not become an SMF topic.
snapshotMode

The criteria for running a snapshot upon startup of the Micro-Integration. The allowed values are:

  • initial—The default. Takes a snapshot on first startup, then streams changes.

  • always—Performs a snapshot on every startup.

  • initial_only—Takes a snapshot but doesn't transition to streaming.

  • no_data—Schema structure only, no data.

  • never—Never takes snapshots.

  • recovery—Takes a snapshot when no offsets are available.

  • when_needed—Takes a snapshot when the Micro-Integration determines it's needed.

  • configuration_based—Uses a Micro-Integration-specific configuration.

  • custom—Uses a custom snapshotter implementation.
skippedOperations

A comma-separated list of operations to skip during streaming. The allowed values are: 

  • c—Skip create/insert operations.

  • u—Skip update operations.

  • d—Skip delete operations.

  • t—Skip truncate operations (default).

  • none—Skip nothing (process all operations).

For example, skippedOperations=t,d indicates to skip truncations and deletions.
tombstonesOnDelete Indicates whether delete operations should be represented by a delete event and a subsequent tombstone event. When tombstonesOnDelete=true, a delete operation generates two events: a delete event and a subsequent tombstone event (key with null value). This allows message brokers with log compaction to completely remove all events for a deleted row's key.
timePrecisionMode

Times, dates, and timestamps can be represented with different kinds of precision, including:

  • adaptive—The default. Captures time and timestamp values exactly as in the database using millisecond, microsecond, or nanosecond precision values based on the database column's type.
  • adaptive_time_microseconds—Always uses microsecond precision for time fields.
  • connect—Always represents time and timestamp values using millisecond precision regardless of the database column's precision.
decimalHandlingMode

Specifies how decimal and numeric columns should be represented in change events. Valid options include:

  • precise—Represents numbers precisely in binary form using java.math.BigDecimal values.
  • double—Represents numbers using Java double values, which may result in a loss of precision but are easier to use.
  • string—Encodes numbers as formatted strings. Semantic information about the real type is lost.