Configuring Connection Details

This section provides instructions for configuring the connection details required to establish communication between the Micro-Integration and your third-party system.

For information about configuring the connection to the event broker, see Connecting to Your Event Broker.

The name of the binder for Azure Blob Storage is azurestorageblob.

Solace Connection Details

The Spring Cloud Stream Binder for Solace uses Spring Boot Auto-Configuration for the Solace Java API to configure its session. In the application.yml, this typically is configured as follows:

solace:
  java:
    host: tcp://localhost:55555
    msg-vpn: default
    client-username: default
    client-password: default    

For more information and options to configure the Solace session, see Spring Boot Auto-Configuration for the Solace Java API.

Preventing Message Loss when Publishing to Topic-to-Queue Mappings

If the Micro-Integration is publishing to a topic that is subscribed to by a queue, messages may be lost if they are rejected (for example, if queue ingress is shut down).

To prevent message loss, configure the reject-msg-to-sender-on-discard option with the including-when-shutdown flag.

Azure Blob Storage Connection Details

To configure the Azure Blob Storage connection details, set the following values in the application.yml file:

spring:
  cloud:
    stream:
      bindings:
        <inputname>:
          destination: "azure-storage-blob://accountName/containerName"
          binder: azurestorageblob

The Spring Cloud Stream standard properties for the Azure Blob Storage are as follows.

Configuration Option	Type	Valid Values	Description
spring.cloud.stream.bindings.<inputname>.destination	String	A valid Azure Blob Storage container name	The Azure Blob Storage account and container name, in the following format: azure-storage-blob://<accountName>/<containerName>
spring.cloud.stream.bindings.<inputname>.binder	String	azurestorageblob	This property must be set to azurestorageblob.

Checkpoint Store Configuration

The Azure Blob Storage stores the current progress of file processing in a checkpoint store backed by a Last Value Queue (LVQ) on a Solace event broker.

The following table lists the configuration options for the checkpoint store.

Configuration Option	Type	Valid Values	Default Value	Description
camel-binder.checkpoint.lvqName	String	Any	None	Required. The name of the LVQ (spool size 0) to be used for checkpointing. The queue must exist on the same event broker and Message VPN as the target queue. If the LVQ is deleted (administratively) or a message from the LVQ is deleted or consumed by another consumer, the Micro-Integration will not be able to resume from the last checkpoint. In addition, the LVQ should not be shared by multiple instances of the Micro-Integration.
camel-binder.checkpoint.autoProvisionLvq	boolean	(true | false)	false	Optional. Set to true to auto-provision the LVQ if it does not exist.

Delete-After-Read Configuration

Configuration Option	Type	Valid Values	Default Value	Description
camel-binder.deleteAfterRead	boolean	true, false	false	When true, the Micro-Integration consumes and deletes the file, regardless of whether it has been read before. When false: If a file has already been consumed, the Micro-Integration periodically downloads it again but ignores it. The time it takes to cycle through the entire folder increases as more files are added. If the Micro-Integration is shut down while a file is pending, it may hang for a short time. Renaming a file will cause it to be consumed again.

Azure Blob Storage Consumer Configuration Options

The following configuration options are available for the Azure Blob Storage consumers. These properties are to be prefixed with spring.cloud.stream.camel.bindings.<inputname>.consumer.endpoint.query-parameters for source bindings.

Configuration Option	Type	Valid Values	Default Value	Description
credentialType	Enum	SHARED_ACCOUNT_KEY SHARED_KEY_CREDENTIAL AZURE_IDENTITY AZURE_SAS	AZURE_IDENTITY	Required. Determines the credential strategy to adopt.
accessKey	String	Any	RAW(${AZURE_ACCESS_KEY})	Required. An access key for the associated Azure account name to be used for authentication with Azure Blob Storage services. The RAW() function is required to supply the key directly.
prefix	String	A valid object key prefix	None	Optional. Specifies a prefix to filter objects in the bucket. Only objects with keys starting with this prefix will be consumed.
fileType	String	json, xml, delimited	None	The file type that determines how the connector parses the content of the consumed files. For details about configuring these file types, see one of: JSON File Type Configuration Options XML File Type Configuration Options Delimited File Type Configuration Options

JSON File Type Configuration Options

When fileType is set to json, the following configuration options are available:

Config Option	Type	Valid Values	Default Value	Description
jsonPath	String	A valid JSONPath expression	$	A JSONPath expression used to split the JSON file into multiple messages. The default is $, which means the entire file is treated as a single message. For example, if a JSON file contains an array of objects [{"a":1}, {"a":2}], setting jsonPath to $.* will generate two separate messages.

XML File Type Configuration Options

When fileType is set to xml, the following configuration options are available:

Config Option	Type	Valid Values	Default Value	Description
xPath	String	A valid XPath expression	/	An XPath expression that splits the XML file into multiple messages. Default is /, which means the entire file is treated as a single message. For example, if an XML file is <root><item>1</item><item>2</item></root>, setting xPath to /root/item generates two messages.

Delimited File Type Configuration Options

When fileType is set to delimited, the following configuration options are available:

Config Option	Type	Valid Values	Default Value	Description
eventDelimiter	String	Any character or string	\n	The character or string that separates records in a delimited file.
paramDelimiter	String	Any character or string	,	The character or string that separates fields within each record.

Advanced Configuration Options

The following additional options are available to control the behavior of the Micro-Integration for Azure Blob Storage. These properties are to be prefixed with spring.cloud.stream.camel.bindings.<inputname>.consumer.endpoint.query-parameters.

Option	Type	Default	Description
blobName	String	None	The name of the blob to consume from a container. On the producer, this parameter is required only for blob-level operations.
blobOffset	long	0	The blob offset for upload or download operations.
closeStreamAfterRead	boolean	true	When true, indicates that the stream closes after a read operation.
dataCount	Long	None	The number of bytes to include in the range. This value must be greater than or equal to 0 if you specify it.
fileDir	String	None	The file directory where downloaded blobs are saved. You can use this parameter in both producer and consumer configurations.
maxResultsPerPage	Integer	None	The maximum number of blobs to return, including all BlobPrefix elements. If the request does not specify maxResultsPerPage or specifies a value greater than 5,000, the server returns up to 5,000 items.
maxRetryRequests	int	0	The maximum number of additional HTTP Get requests that are made while reading data from a response body.
prefix	String	None	A filter that returns only blobs whose names begin with the specified prefix. This parameter can be null to return all blobs.
regex	String	None	A filter that returns only blobs whose names match the specified regular expression. This parameter can be null to return all blobs. If you set both prefix and regex, regex takes priority and prefix is ignored.
sasToken	String	None	A Shared Access Signature (SAS) token for authentication with Azure Blob Storage.
blobSequenceNumber	Long	0	A user-controlled value that you can use to track requests. The sequence number value must be between 0 and 263 - 1.
closeStreamAfterWrite	boolean	true	When true, indicates that the stream closes after a write operation.
commitBlockListLater	boolean	true	When true, indicates that staged blocks are not committed directly.
createAppendBlob	boolean	true	When true, indicates that append blocks are created when committing append blocks.
createPageBlob	boolean	true	When true, indicates that the page blob is created when uploading a page blob.
downloadLinkExpiration	Long	None	The expiration time in milliseconds that overrides the default expiration for the URL download link.
pageBlobSize	Long	512	The maximum size for the page blob, up to 8 TB. The page blob size must be aligned to a 512-byte boundary.
backoffErrorThreshold	int	None	The number of subsequent error polls that must occur before the backoffMultiplier activates. Error polls are polls that fail because of an error.
backoffIdleThreshold	int	None	The number of subsequent idle polls that must occur before the backoffMultiplier activates.
backoffMultiplier	int	None	A multiplier that allows the scheduled polling consumer to back off if there has been a number of subsequent idle polls or errors in a row. The multiplier represents the number of polls that are skipped before the next actual attempt occurs. When you use this option, you must also configure backoffIdleThreshold and/or backoffErrorThreshold.
delay	long	500	The number of milliseconds before the next poll.
greedy	boolean	false	When true, indicates that the ScheduledPollConsumer runs immediately again if the previous run polled one or more messages.
initialDelay	long	1000	The number of milliseconds before the first poll starts.
repeatCount	long	0	The maximum number of times the scheduler fires. For example, if you set this parameter to 1, the scheduler fires only once. If you set it to 5, the scheduler fires five times. A value of zero or a negative value means the scheduler fires indefinitely.
useFixedDelay	boolean	true	When true, indicates that fixed delay is used rather than fixed rate. See ScheduledExecutorService in the JDK for details.
sourceBlobAccessKey	String	None	The access key for the source blob in a copy blob operation. Because passing an access key as a header is unsafe, you can set it as a parameter instead.
credentials	Object	None	Credentials object for authentication with Azure Blob Storage.
credentialType	String	None	The type of credential to use for authentication (e.g., AZURE_IDENTITY, AZURE_SAS, SHARED_ACCOUNT_KEY).

Connecting to Multiple Systems

To connect to multiple systems of a same type, use the multiple binder syntax.

For example:

spring:
  cloud:
    stream:
      binders:
      
      # 1st solace binder in this example
      solace1:
        type: solace
        environment:
          solace:
            java:
              host: tcp://localhost:55555

      # 2nd solace binder in this example
      solace2:
        type: solace
        environment:
          solace:
            java:
              host: tcp://other-host:55555

        # The only azure-blob binder
        azure-blob1:
          type: azurestorageblob
          # Add `environment` property map here if you need to customize this binder.
          # But for this example, we'll assume that defaults are used.

        # Required for internal use
        undefined:
          type: undefined
      bindings:
        input-0:
          destination: <input-destination>
          binder: azure-blob1
        output-0:
          destination: <output-destination>
          binder: solace1 # Reference 1st solace binder
        input-1:
          destination: <input-destination>
          binder: azure-blob1
        output-1:
          destination: <output-destination>
          binder: solace2 # Reference 2nd solace binder

The configuration above defines two binders of type solace and one binder of type azurestorageblob, which are then referenced within the bindings.

Each binder above is configured independently under spring.cloud.stream.binders.<bindername>.environment..