PostgreSQL CDC

The Micro-Integration for PostgreSQL CDC captures changes (inserts, updates, deletes) from a PostgreSQL database table (using Debezium) and streams them to a topic on an event broker service.

You can configure PostgreSQL only as a source.

To configure the connection details for PostgreSQL in your Micro-Integration, see PostgreSQL CDC Connection Parameters

You must also define at least one Micro-Integration Flow that specifies:

For message headers, see PostgreSQL CDC Message Headers.

PostgreSQL CDC Connection Parameters

The following table describes the connection parameters for PostgreSQL CDC.

Field Description
Database Hostname The hostname of the PostgreSQL database server.
Database Port The port of the PostgreSQL database server.
Database Name The name of the PostgreSQL database.
Skip Delete Events Select to skip the processing of delete events from the database change stream.
Authentication

The type of authentication to use when connecting to PostgreSQL. Choose one of:

Additional Properties

(Optional) Any additional configuration related to PostgreSQL that applies for all Flows. Click Add Context if you need to specify additional context properties.
Checkpoint Queue The name of the Last Value Queue (LVQ) on the Solace event broker to use for checkpointing. The LVQ stores information about the files that have been processed and the files that are currently being processed. The LVQ must exist on the same event broker and VPN as the target destination.
Auto Create Checkpoint Queue Select to automatically create the LVQ if it does not exist. The default value is false.

Basic Authentication Parameters

Field Description
Username The username to use for logging in to PostgreSQL.
Password The password to use for authenticating with the PostgreSQL database.

Client Certificate Authentication Parameters

Field Description
Database SSL mode

The SSL mode determines the security level for client-server communications. It controls whether SSL/TLS encryption is used and how rigorously the server's identity is verified. Choose one of:

  • disable to use an unencrypted connection

  • allow to try to use an unencrypted connection first and, failing that, a secure (encrypted) connection

  • prefer to try to use a secure (encrypted) connection first and, failing that, an unencrypted connection

  • require to use a secure (encrypted) connection, and fail if one cannot be established

  • verify-ca like required but additionally verify the server TLS certificate against the configured Certificate Authority (CA) certificates, or fail if no valid matching CA certificates are found

  • verify-full like verify-ca but additionally verify that the server certificate matches the host to which the connection is attempted

For more information, see the Protection Provided in Different Modes section of the PostgreSQL documentation.
Client certificate The SSL Certificate for the client.
Client private key The SSL private key for the client.
Private key password The database SSL password to authenticate with the PostgreSQL database.
SSL Root Certificate The database SSL root certificate against which the server is validated.
Username The username to use for logging in to PostgreSQL.

Micro-Integration Flow Parameters

You must configure the endpoint parameters for each Flow. Each Flow can have different settings, but they all share the connection details of the parent Micro-Integration.

PostgreSQL CDC Source Parameters

The following table describes the parameters for configuring PostgreSQL CDC as a source.

Field Description
Destination The name of the table, including the schema name. Specify the destination using the <schemaName>.<tableName> format.
Slot Name

The name of the PostgreSQL logical decoding slot.

A logical decoding slot is a persistent object that maintains a stream of database changes. The Micro-Integration for PostgreSQL CDC consumes the change stream from the specified slot.

Additional Properties

(Optional) Any additional configuration related to PostgreSQL that apply only for this Flow. Click Add Context if you need to specify additional context properties.

Additional Properties

The following table lists additional properties that are available.

Property Type Valid Values Default Value Description

slotName

String

The name of the PostgreSQL logical decoding slot. A logical decoding slot is a persistent object that maintains a stream of database changes from a plugin. The Micro-Integration for PostgreSQL CDC consumes the change stream from the specified slot. For more information, consult your PostgreSQL server administrator.

topicPrefix

String

The binding name (for example, input-0)

Topic prefix that identifies and provides a namespace for the particular database server/cluster that is capturing changes.

snapshotMode

String

no_data | always | initial | initial_only | never | custom

no_data

The criteria for running a snapshot upon startup of the Micro-Integration.

pluginName

String

pgoutput | decoderbufs

pgoutput

The name of the PostgreSQL logical decoding plugin installed on the server. The logical decoding plugin converts write-ahead log (WAL) entries into a stream of changes. For more information, consult your PostgreSQL server administrator.

skippedOperations

String

t

A comma-separated list of operations to skip during streaming. Valid options include: c for inserts/create; u for updates; d for deletes, t for truncates, and none to indicate nothing skipped.

publicationName

String   dbz_publication The name of the PostgreSQL publication used for streaming changes from a plugin.

publicationAutocreateMode

String

disabled | all_tables | filtered

disabled

Specifies when a publication is to be created. Applies only when streaming changes using pgoutput. Select one of the following values:

  • disabled—The Micro-Integration will not attempt to create a publication at all. The expectation is that the user has created the publication up-front. If the publication isn’t found to exist upon startup, the Micro-Integration will throw an exception and stop.

  • all_tables—If no publication exists, the Micro-Integration will create a new publication for all tables:

    CREATE PUBLICATION FOR ALL TABLES;

    Note this requires that the configured user has access. If the publication already exists, it will be used.

  • filtered—If no publication exists, the Micro-Integration will create a new publication for all those tables matching the current filter configuration:

    CREATE PUBLICATION FOR TABLE;

    If the publication already exists, it will be used.

decimalHandlingMode

String precise | string | double precise

Specify how decimal and numeric columns should be represented in change events, including:

  • precise—Uses java.math.BigDecimal to represent values.

  • string—Uses string to represent values.

  • double—Represents values using Java’s double, which may not offer the precision but will be far easier to use in consumers.

timePrecisionMode

String adaptive | adaptive_time_microseconds | connect adaptive

Time, date, and timestamps can be represented with different kinds of precisions, including:

  • adaptive—Bases the precision of time, date, and timestamp values on the database column’s precision.

  • adaptive_time_microseconds—Like adaptive mode, but time fields always use microseconds precision.

  • connect—Uses millisecond precision regardless of the database columns' precision.

tombstonesOnDelete

boolean

false

Indicates whether delete operations should be represented by a delete event and a subsequent tombstone event.

tableIncludeList

This parameter is ignored.

schemaIncludeList

This parameter is ignored.

offsetStorageTopic

This parameter is ignored.

offsetStoragePartitions

This parameter is ignored.

offsetStorageReplicationFactor

This parameter is ignored.