IBM Db2 Bulk CDC

The Micro-Integration for IBM Db2 Bulk CDC captures changes (inserts, updates, deletes) from multiple IBM Db2 database tables simultaneously (using Debezium) and streams them to topics on an event broker service. Only header mapping is supported; payload mapping is not available for this Micro-Integration.

You can configure IBM Db2 only as a source.

To configure the connection details for IBM Db2 in your Micro-Integration, see IBM Db2 Bulk CDC Connection Parameters.

You must also define one Micro-Integration Flow that specifies:

  • the endpoints where data is consumed from and published to. For more information, see Micro-Integration Flow Parameters.

  • how the header fields should be mapped. For more information, see Mapping Headers and Payloads.

    For bulk CDC Micro-Integrations, payload transformations are not supported. Header transformations and Smart Topics are supported.

For message headers, see IBM Db2 Bulk CDC Message Headers .

IBM Db2 Bulk CDC Connection Parameters

The following table describes the connection parameters for IBM Db2.

Field Description
Database Hostname The hostname of the IBM Db2 database server.
Database Port The port of the IBM Db2 database server.
Database Name The name of the IBM Db2 database.
Authentication

The authentication scheme to use when connecting to IBM Db2. Select one of:
Skip Delete Events Select to skip the processing of delete events from the database change stream. The option is enabled by default.
Additional Properties

(Optional) Any additional configuration related to IBM Db2. Click Add Context if you need to specify additional context properties. For more information, see Additional Properties.
Checkpoint Queue The name of the Last Value Queue (LVQ) on the event broker service to use for checkpointing. The LVQ stores information about the change events that have been processed and those that are currently being processed. The LVQ must exist on the same event broker service and VPN as the target destination.
Auto Create Checkpoint Queue Select this option to automatically create the LVQ if it does not exist.

Basic Authentication Parameters

Field Description
Username The username to use for logging in to IBM Db2.
Password The password to use for authenticating with the IBM Db2 database.

SSL Certificate Authentication (Db2) Parameters

Field Description
SSL Truststore The truststore file containing trusted certificates for the IBM Db2 server.
Truststore Type The type of the SSL truststore, either JKS or PKCS12.
Truststore Password The password for the SSL truststore.
Username The username to use for logging in to IBM Db2.
Password The password to authenticate with the IBM Db2 database.

Micro-Integration Flow Parameters

You must configure the endpoint parameters for the Flow. The Flow shares the connection details of the parent Micro-Integration.

The following table describes the parameters for configuring IBM Db2 as a source.

Field Description
Destination

The tables from which to consume change events. Each entry must include the database name and the table name in <database>.<table> format. You can specify multiple tables using a semicolon-separated list, a regular expression, or a combination of both. For example:

  • Semicolon-separated list: INVENTORY.CUSTOMERS;INVENTORY.ORDERS

  • Regular expression: INVENTORY.TABLE_.*

  • Combination: inventory.orders_.*;inventroy.customers_.*
DB2 Boolean Column Selector Regular expression used to select SMALLINT columns that should be interpreted as boolean values. This is optional and should only be set when SMALLINT columns represent boolean semantics.
Additional Properties

(Optional) Any additional configuration related to IBM Db2 that applies for this Flow. Click Add Context if you need to specify additional context properties. For more information, see Additional Properties.

Additional Properties

The following table lists additional properties that are available.

Property Default Value Description

snapshotMode

no_data

Controls initial snapshot behavior. Select one of the following snapshot options:

  • initial—The Micro-Integration performs a database snapshot as described in the default workflow for creating an initial snapshot. After the snapshot completes, the Micro-Integration begins to stream event records for subsequent database changes.

  • initial_only—The Micro-Integration performs a database snapshot. After the snapshot completes, the Micro-Integration stops, and does not stream event records for subsequent database changes.

  • schema_only—Deprecated, see no_data.

  • never, no_data—The Micro-Integration does not run a snapshot. Upon first startup, the Micro-Integration immediately begins reading from the beginning of the log, but it does not create READ events to represent the data set at the point of the Micro-Integration's startup.

  • when_needed—After the Micro-Integration starts, it performs a snapshot only if it detects one of the following circumstances:

    • It cannot detect any topic offsets.

    • A previously recorded offset specifies a log position that is not available on the server.

maxBatchSize

2048

The maximum number of events to batch.

maxQueueSize

8192

The maximum size of the queue for change events read from the database log but not yet recorded or forwarded. Always set this value to be larger than the maximum batch size.
includeSchemaChanges false When true, indicates that the Micro-Integration captures DDL schema changes.
tombstonesOnDelete true Controls whether a delete event is followed by a tombstone event:

  • true—A delete operation is represented by a delete event and a subsequent tombstone event.

  • false—Only a delete event is emitted.

eventProcessingFailureHandlingMode

fail

Specifies how the Micro-Integration handles exceptions during processing of events. The possible values are:

  • fail—The Micro-Integration logs the offset of the problematic event and stops processing.

  • warn—The Micro-Integration logs the offset of the problematic event and continues processing with the next event.

  • skip—The Micro-Integration skips the problematic event and continues processing with the next event.
schemaNameAdjustmentMode None

Controls how schema and table names are adjusted to comply with serialization constraints:

  • avro_unicode—replaces the underscore or characters that cannot be used in the Avro type name with corresponding unicode like _uxxxx. (Note: _ is an escape sequence like backslash in Java).

  • none—does not apply any adjustment (default).
decimalHandlingMode double

Specifies how DECIMAL and NUMERIC types are represented in change events:

  • precise—Uses java.math.BigDecimal to represent values, which are encoded in the change events using a binary representation and Kafka Connect’s org.apache.kafka.connect.data.Decimal type.

  • string—Uses string to represent values.

  • double—Represents values using Java's double, which may not offer the same precision but is easier to use in consumers.
timePrecisionMode adaptive

Determines how temporal types are represented in emitted events. Values include:

  • adaptive—Bases the precision of time, date, and timestamp values on the database column's precision.

  • connect—Represents time, date, and timestamp values using millisecond precision regardless of the database columns' precision.
tableExcludeList null A comma-separated list of regular expressions that match the fully-qualified names of tables to be excluded from monitoring.

skippedOperations

t, d

A comma-separated list of the operation types that you want the Micro-Integration to skip during streaming. You can configure the Micro-Integration to skip the following types of operations:

  • c (insert/create)

  • u (update)

  • d (delete)

  • t (truncate)
schemaHistoryInternalStoreOnlyCapturedTablesDdl true Property that controls the Debezium DDL; if set to false, it captures all available tables.