Substitution Expressions Overview

Substitution expressions are a Solace-specific expression language used to replace specific text attributes (request targets, request headers, etc.) with system generated output.

For example, suppose your deployment warrants using REST Delivery Points (RDPs) to send messages into storage services such as Azure Storage, Amazon S3, or Google Cloud Storage. This requires sending each message to a unique target, preferably with that uniqueness coming from the message topic, metadata, or the current time. To enable this, you could embed a substitution expression into the request target that configures the system to modify the target based on the data (topic, current time, etc.) from each incoming message.

A substitution expression consists of either a literal or a function call. Function call parameters can be any substitution expression that is of the appropriate type. When embedded in text attributes, substitution expressions are delimited by ${ and }. Only close braces outside of string literals end a substitution expression; there is no special escaping of braces required within a substitution expression. To include ${ in the text, a substitution expression can be used. For example, the ${ can be encoded as ${"$"}{.

Any substitution expression that isn't valid results in an error at configure time. You can view error message details through all available monitoring interfaces (Broker Manager, CLI, SEMPv1, and SEMPv2).

Substitution Expression Functions

Substitution expression functions return a value and possibly take parameters. The currently available functions and whether they are applicable to RDP, Kafka sender, or Kafka receiver objects, are listed in the following table.

Function parameters and return values have specified types. A string may be passed in anywhere bytes are accepted; there are no other automatic conversions. The available types are:

integer: a 64-bit integer value (range: -9223372036854775808 through 9223372036854775807).
string: a UTF-8 encoded string.
bytes: a sequence of 0 or more bytes.

The integer timestamp parameter passed to all the time functions (utcYear, utcDate etc.) must be in milliseconds.

Function Name	Usage	Parameter Types	Return Type	Context			Description
				RDP	Kafka
				RDP	Sender	Receiver
topic	`topic([N])`	integer	string				Returns part or all of the message topic. If N is positive, returns the Nth field, numbering from 1. If N is negative, returns the Nth field from the end of the topic. If N is 0 or not specified, returns the entire topic string. If the selected field doesn't exist, returns the empty string. For example, if the topic was `alfa/bravo/charlie/delta`, then `topic(2)` would return `bravo`, `topic(-2)` would return `charlie`, `topic()` or `topic(0)` would return `alfa/bravo/charlie/delta`, and `topic(8)` or `topic(-7)` would return the empty string.
kafkaTopic	`kafkaTopic([N, separator])`	integer, string	string				Returns the topic from a Kafka message. If N is provided, separator will be used as the topic level separator. The separator is required as Kafka does not define this (although by convention dot and underscore are common).
now	`now()`	N/A	integer				Returns the current time in milliseconds from midnight, January 1, 1970 UTC. For example, if the time were Mon 22 Jun 2020 13:13:02.123 EDT, the value returned would be `1592845982123`. Note that due to implementation limitations, the accuracy and resolution of the milliseconds may be low. It is guaranteed that the `now()` value will never decrease unless the clock on the event broker is updated or corrected.
senderTimestamp	`senderTimestamp()`	N/A	integer				Returns the `senderTimestamp` property of the message in milliseconds from midnight, January 1, 1970 UTC, or 0 if the `senderTimestamp` property is not set (which means a timestamp of midnight, January 1, 1970 UTC).
kafkaTimestamp	`kafkaTimestamp()`		integer				Returns the timestamp message field from the Kafka message.
ts	`ts()`	N/A	integer				If the `senderTimestamp` property is present on the message, the value of `senderTimestamp()` is returned, otherwise the value of `now()` is returned.
utcDate	`utcDate([separator [, timestamp]])`	string, integer	string				Returns the date portion of the timestamp in UTC, formatted as `YYYY-MM-DD`. If provided, separator is used instead of hyphens to separate the parts. If timestamp is omitted, the value returned by `ts()` is used.
utcYear	`utcYear([timestamp])`	integer	string				Returns the four digit UTC year. If timestamp is omitted, the value returned by `ts()` is used.
utcMonth	`utcMonth([timestamp])`	integer	string				Returns the two digit UTC month. If timestamp is omitted, the value returned by `ts()` is used.
utcDay	`utcDay([timestamp])`	integer	string				Returns the two digit UTC day. If timestamp is omitted, the value returned by `ts()` is used.
utcTime	`utcTime([separator [, timestamp]])`	string, integer	string				Returns the time portion of the timestamp in UTC, formatted as `hh:mm:ss`. If provided, separator is used instead of colons to separate the parts. If timestamp is omitted, the value returned by `ts()` is used.
utcHour	`utcHour([timestamp])`	integer	string				Returns the two digit UTC hour. If timestamp is omitted, the value returned by `ts()` is used.
utcMinute	`utcMinute([timestamp])`	integer	string				Returns the two digit UTC minute. If timestamp is omitted, the value returned by `ts()` is used.
localDate	`localDate([separator [, timestamp]])`	string, integer	string				Returns the date portion of the timestamp in the broker's local time zone, formatted as `YYYY-MM-DD`. If provided, separator is used instead of hyphens to separate the parts. If timestamp is omitted, the value returned by `ts()` is used.
localYear	`localYear([timestamp])`	integer	string				Returns the four digit year portion of the specified timestamp in the broker's local time zone. If timestamp is omitted, the value returned by `ts()` is used.
localMonth	`localMonth([timestamp])`	integer	string				Returns the two digit month portion of the specified timestamp in the broker's local time zone. If timestamp is omitted, the value returned by `ts()` is used.
localDay	`localDay([timestamp])`	integer	string				Returns the two digit day portion of the specified timestamp in the broker's local time zone. If timestamp is omitted, the value returned by `ts()` is used.
localTime	`localTime([separator [, timestamp]])`	string, integer	string				Returns the time portion of the timestamp in the broker's local time zone, formatted as `hh:mm:ss`. If provided, separator is used instead of colons to separate the parts. If timestamp is omitted, the value returned by `ts()` is used.
localHour	`localHour([timestamp])`	integer	string				Returns the two digit hour portion of the specified timestamp in the broker's local time zone. If timestamp is omitted, the value returned by `ts()` is used.
localMinute	`localMinute([timestamp])`	integer	string				Returns the two digit minute portion of the specified timestamp in the broker's local time zone. If timestamp is omitted, the value returned by `ts()` is used.
second	`second([timestamp])`	integer	string				Returns the two digit second portion of the specified timestamp. If timestamp is omitted, the value returned by `ts()` is used.
millisecond	`millisecond([timestamp])`	integer	string				Returns the three digit millisecond portion of the specified timestamp. If timestamp is omitted, the value returned by `ts()` is used.
unixTime	`unixTime([timestamp])`	integer	integer				Returns the number of seconds since midnight, January 1, 1970 UTC for the specified timestamp, ignoring leap seconds. This is known as unix time or epoch time. For example, if the provided timestamp was for Mon 22 Jun 2020 13:13:02.123 EDT, the value returned would be `1592845982`. If timestamp is omitted, the value returned by `ts()` is used.
randomBytes	`randomBytes(N)`	integer	bytes				Returns N cryptographically random bytes. N must be a positive integer less than 100.
uuid UUID	`uuid()` `UUID()`	N/A	string				Returns a new UUID. The `uuid()` function generates lowercase letters; the `UUID()` function generates uppercase letters. For example, `UUID()` might return the string `D64DB3F0-63A8-4D9B-B605-FC30B5216552`. Each time these functions are called, a different UUID is generated.
base32 BASE32 base32NoPad BASE32NoPad	`base32(data)` `BASE32(data)` `base32NoPad(data)` `BASE32NoPad(data)`	bytes	string				Returns the provided data (which can be of type string or bytes) encoded as base32. There are four variants, for the four combinations of uppercase/lowercase and padding. The variants that start with base32 will produce lowercase output; the variants that start with BASE32 will product uppercase output. The variants that end with `NoPad` do not add padding.
base64 base64NoPad	`base64(data)` `base64NoPad(data)`	bytes	string				Returns the provided data (which can be of type string or bytes) encoded as base64. The `base64NoPad` variation omits padding.
base64Url base64UrlNoPad	`base64Url(data)` `base64UrlNoPad(data)`	bytes	string				Returns the provided data (which can be of type string or bytes) encoded as base64url. The `base64UrlNoPad` variation omits padding.
hex HEX	`hex(data)` `HEX(data)`	bytes	string				Return the provided data (which can be of type string or bytes) encoded as hexadecimal (base16). The `hex` variant produces lowercase output and the `HEX` variant produces uppercase output.
urlEncode	`urlEncode(data [, exceptions])`	bytes, string	string				Returns the provided data (which can be of type string or bytes) URL encoded: any characters that are not unreserved characters according to RFC 3986 section 2.3 are percent-encoded (with the hexadecimal digits A-F in uppercase as per that specification), unless the character is specified in the exceptions string. If exceptions is not provided, it defaults to the empty string.
senderId	`senderId()`	N/A	string				Returns the `senderId` property of the message, or the empty string if the `senderId` property is not set.
userPropertyAsString	`userPropertyAsString(propertyName)`	string	string				Returns the propertyName (a case-sensitive string) user property value as a string. Returns the empty string if there is no user property with the specified name. If there is more than one user property with the specified name, only one of them is returned (it is undefined which one).
kafkaHeader	`kafkaHeader(headerName)`	string	bytes				Gets a header field from the Kafka message. Behaviours for missing or duplicated headerName is the same as `userPropertyAsString()`.
kafkaHeaderAsString	`kafkaHeaderAsString(headerName)`	string	string				Returns the same vale as `kafkaHeader()`, but with bytes converted to a string. Each byte is interpreted as an ASCII character, truncated at the first non-printable character.
msgType	`msgType()`	N/A	string				Returns the Solace Application Message Type (a string). Note that there are several different things that message type can refer to. In this context it is an arbitrary value set by the client and passed along but otherwise ignored by the broker. In JMS it is accessed as `JMSType`; in CCSMP it can be set via `solClient_msg_setApplicationMsgType`.
msgId	`msgId()`	N/A	string				Returns the message ID (a string).
sequenceNumber	`sequenceNumber()`	N/A	integer				Returns the sequence number (an integer) of the message.
correlationId	`correlationId()`	N/A	string				Returns the correlation ID (a string) of the message.
partitionKey	`partitionKey()`	N/A	string				Returns the partitioned queue key of the message.
kafkaPartitionKey	`kafkaPartitionKey()`	N/A	bytes				Returns the Kafka partition key of the message as bytes.
kafkaPartitionKeyAsString	`kafkaPartitionKeyAsString()`	N/A	string				Returns the same value as `kafkaPartitionKey()`, but with bytes converted to a string. Each byte is interpreted as an ASCII character, truncated at the first non-printable character.
kafkaPartitionNumber	`kafkaPartitionNumber()`	N/A	integer				Returns the partition number the message was received from.
kafkaPartitionOffset	`kafkaPartitionOffset()`	N/A	integer				Returns the partition offset of the message.
replace	replace(source, old, new [, count])	string, string, string, integer	string				Returns source with string old replaced with new, up to count number of times starting from the first (or last if count is negative). If count is not provided, all occurrences of old are replaced. While of general use, this has been added to convert Kafka Topics (with dots or underscores) to Solace Topics (with slashes).
withDefault	`withDefault(value, default)`	string, string	string				Returns value unless value is the empty string, in which case it returns default.

Substitution Expression Processing Examples

The following table lists several examples that illustrate how PubSub+ event brokers process substitution expressions. The examples assume a message arriving at January 2, 2006 3:04:05.123 pm with the following data:

topic=t/v1/DE/sk/escalator/1,2/sk_1.x/DE12345/raw
sender timestamp=January 2, 2006 3:04:04.987 pm
"author" user property=Jane "The Bear" Doe

In specific contexts in which substitution expressions are enabled, further processing may happen after substitution expression evaluation. For example, after substitution processing request targets will have spaces and non-ASCII characters percent-encoded to ensure that the target is valid.
Line breaks and indentation in the following examples are for readability only.

Input	Output
`${topic()}`	`t/v1/DE/sk/escalator/1,2/sk_1.x/DE12345/raw`
`${urlEncode(topic(), "/")}`	`t/v1/DE/sk/escalator/1%2C2/sk_1.x/DE12345/raw`
`${urlEncode(topic())}`	`t%2Fv1%2FDE%2Fsk%2Fescalator%2F1%2C2%2Fsk_1.x%2FDE12345%2Fraw`
`global/landing/solace/${topic(5)}/${topic(7)}/${topic(8)}/${ts()}`	`global/landing/solace/escalator/sk_1.x/DE12345/1136189044987`
`${utcDate()}/${utcTime()}/${base32(randomBytes(15))}`	`2006-01-02/15:04:04/4fdm3hlyafhh77ntxqcwbt7b`
`${utcDate("/")}/${utcTime("/")}/${BASE32(randomBytes(15))}`	`2006/01/02/15/04/04/4FDM3HLYAFHH77NTXQCWBT7B`
`${userPropertyAsString("author")}`	`Jane "The Bear" Doe`
`${"${"}text in the delimiters}`	`${text in the delimiters}`
`A number: ${5}`	`A number: 5`
`${"${This is a valid substitution expression.}"}`	`${This is a valid substitution expression.}`
`${"This is also a valid substitution expression! :-}"}`	`This is also a valid substitution expression! :-}`
`This is a valid attribute value with no substitution expression. :-}`	`This is a valid attribute value with no substitution expression. :-}`
`${base64("Strings can be used where bytes are specified.")}`	`U3RyaW5ncyBjYW4gYmUgdXNlZCB3aGVyZSBieXRlcyBhcmUgc3BlY2lmaWVkLg==`

Text Attributes That Can Contain Substitution Expressions

The following text attributes can contain substitution expressions:

RDP POST request targets, if substitution expression evaluation is configured. For more information, see Managing REST Delivery Points.
RDP request headers. For more information, see Managing REST Delivery Points.
Kafka receiver topic bindings (SMF topic generation). For more information, see Configuring SMF Topic Generation.
Kafka receiver topic bindings (partitioned queue key generation). For more information, see Configuring Partition Key Generation.
Kaka sender queue bindings (Kafka partition key generation). For more information, see Configuring Kafka Partition Key Generation.

Provide feedback