Mapping Message Headers and Payloads

You can map header and payload values between your source and target systems. Mapping allows you to define the relationships between the data fields in your source system and the data fields in your target system. If you don't explicitly map any source headers, their values are not propagated to the target. If you don't map any payload fields, the payload is passed through to the target system unchanged.

You map header and payload fields between your source and target systems using an expression language based on Spring Expression Language (SpEL). In addition, you can use transformation functions to modify the source header or payload data so that it conforms to your target format.

The mappings are specified in the transform configuration section of the application.yml file, as shown below, where:

• <workflow-id> is a value between 0 and 19

• <expression> is an expression that defines a transformation between the source and target message

solace:
  connector:
    workflows:
      <workflow-id>:
        enabled: true
        transform:
          source-payload:
            content-type: "application/json"
          target-payload:
            content-type: "application/json"
          expressions:
            - transform: <expression>
            - transform: <expression>

The properties under the solace.connector.workflows.<workflow-id>.transform prefix are as follows:

• source-payload.content-type, target-payload.content-type—The content type to use to interpret the source payload or target payload, respectively. For more information, see Payload Content Types.

• expressions—An ordered list of transformation expressions to apply to the message. To learn more about expressions, see Writing Transformation Expressions.

For a detailed example that shows how to map and transform the headers and payload a specific message, see Mapping and Transformation Example.

Payload Content Types

The source-payload.content-type and target-payload.content-type properties define how the source and target payloads are interpreted and serialized.

Content types are not data types. Content types determine how the physical data is logically interpreted by the Micro-Integration. For example, a JSON payload can be interpreted from any payload whose data type is a string, a byte array, or a map.

The following content types are supported:

Content Type	Content Type Category	Description
application/vnd.solace.​micro-integration.unspecified	Unspecified	The content type is unspecified and cannot be read nor modified.
application/json	Structured	The content type is interpreted/serialized as JSON.
application/xml	Structured	The content type is interpreted/serialized as XML.

If not specified, the content type defaults to application/vnd.solace.​micro-integration.unspecified.

The source-payload.content-type and target-payload.content-type are handled as separate and independent operations:

• The source-payload.content-type setting determines how the source payload is read and interpreted. For example, when the content type is set to application/json, the source payload is expected to be JSON.

• The target-payload.content-type setting determines how the target payload is interpreted and serialized. For example, when the content type is set to application/xml, the target payload is written as XML.

Payload Expressions

Expressions between the following content types are supported:

• JSON to JSON

• XML to XML

• JSON to XML

• XML to JSON

If the content type is JSON or XML, you can write expressions that allow you to drill down into the structure of the source and target payloads.

Element Accessors

You can write expressions to access elements in the payload such as source['payload']['book']['title'] or target['payload']['book']['title']. This expression corresponds to the title node in the following examples:

• JSON:

  {
    "book": {
      "isbn": "978-1-947263-15-4",
      "author": "Evelyn Marlowe",
      "title": "The Quantum Firewall"
    }
  }

• XML:

  <book>
      <isbn>978-1-947263-15-4</isbn>
      <author>Evelyn Marlowe</author>
      <title>The Quantum Firewall</title>
  </book>

Indexed Accessors

You can use indexed accessors such as source['payload']['orderItem'][2]['color'] or target['payload']['orderItem'][2]['color'] to refer to JSON array elements or repeated XML nodes. This expression corresponds to Multicolor in the following examples:

• JSON array elements:

  {
    "order": {
      "orderItem": [
        {
          "name": "Robot Action Figure",
          "color": "Silver"
        },
        {
          "name": "Plush Dinosaur",
          "color": "Green"
        },
        {
          "name": "Building Blocks Set",
          "color": "Multicolor"
        }
      ]
    }
  }

• repeated XML nodes:

  <order>
      <orderItem>
          <name>Robot Action Figure</name>
          <color>Silver</color>
      </orderItem>
      <orderItem>
          <name>Plush Dinosaur</name>
          <color>Green</color>
      </orderItem>
      <orderItem>
          <name>Building Blocks Set</name>
          <color>Multicolor</color>
      </orderItem>
  </order>

  Note that for source payloads, the indexed accessor syntax (for example, [0]) only works when the XML payload contains multiple elements with the same name. If the XML source contains only a single element, the indexed accessor fails with an error.

XML Attributes

To reference XML attributes, you can use the syntax @<attribute_name>. For example, consider the following XML payload:

<item id="123">		
  <name>Laptop</name>
  <price>999.99</price>
</item>

You can use the following expressions to access the attribute in the item element:

• source['payload']['item']['@id'] (for source XML payloads)

• target['payload']['item']['@id'] (for target XML payloads)

To access the text in elements that also have attributes, use the #text keyword.

For example, if you had the following element in your source XML payload:

<course courseCode="MAT1341">Linear Algebra</course> 

And you used the following expressions:

target['payload']['courseName']['#text']=source['payload']['course']['#text']
target['payload']['courseName']['@id'] = source['payload']['course']['@courseCode']

The resulting XML output would be:

<courseName id="MAT1341">Linear Algebra</courseName>

Limitations

Due to their internal representation, there are several limitations in converting source payloads to target payloads:

• Not all external systems associated with Micro-Integrations support XML for transformations. For example:

  • Salesforce as a target requires JSON payloads.

  • Salesforce as a source produces only JSON payloads.

  • All CDC Micro-Integrations require JSON payloads.

• XML comments, processing instructions, and namespaces are dropped if any mappings are present. These elements are preserved only in pass-through scenarios using application/vnd.solace.​micro-integration.unspecified where target['payload'] = source['payload'] is the only expression used.

• Mixed-content XML (that is, XML with elements that contain both text and child elements) is not supported.

• Element ordering is not preserved in transformations from XML when you use payload transformation expressions.

• XML namespace resolution is not supported. You cannot set XML namespaces on output documents, regardless of the transformation expressions you use.

• In XML to JSON mappings, multiple identically-named nodes are translated to JSON arrays, but there is no way to transform a single XML node to a JSON array with a single element. This behavior can affect transformation execution in unexpected ways. A transformation expression with an indexed accessor (as shown in the following example) works as expected when the XML contains several identically-named nodes, but fails with an error if only a single node is present:

  target['payload']['abc'] = source['payload']['books'][0]['author']

• JSON arrays do not have a direct XML equivalent. When you transform from JSON to XML, arrays are represented as repeated elements. This can cause unexpected results.

• In JSON to XML mappings, if the incoming JSON has no root node, a root node named <ObjectNode> is automatically added to the XML payload to satisfy XML's requirement for a single root element.

• JSON property names with special characters may not be valid XML element names. Additionally, naming convention differences between JSON (which typically uses camelCase) and XML may require you to perform additional field transformations.

• XML parsing and generation is slower than JSON processing, which may impact performance in high-throughput scenarios.

Writing Transformation Expressions

To map the data coming from your source system to your target system, you supply a set of expressions in the configuration for each workflow. Specifically, under the solace.connector.workflows.<workflow-id>.transform prefix, you set the following properties:

• expressions, which is an ordered list of transformation expressions to apply to the message.

• Under expressions, one or more instances of transform. Each instance of transform is set to a single expression to transform the message.

A transformation expression can reference:

• ['payload'] to access the message payload.

• ['headers']['<header_name>'] to access a message header value.

• transformation functions, which are built in and can be called directly from transformation expressions.

  To call a function, use the # character followed by the function name. For a detailed list of functions, see the Transformation Function Reference.

The following context variables are available for you to use in your message transformation expressions:

Variable	Access	Description
source['headers']	Read-Only	A map of key-value pairs representing the headers of the source message. For more information, see Header Propagation.
source['payload']	Read-Only	The payload of the source message as interpreted by source-payload.content-type. For more information, see Payload Propagation.
target['headers']	Write-Only	A map of key-value pairs representing the headers of the target message. For more information, see Header Propagation.
target['payload']	Write-Only	The payload of the target message as interpreted by target-payload.content-type. For more information, see Payload Propagation.
var	Read/Write	A named variable that can be used to store intermediate values during a transformation. For more information, see Saving Intermediate Values.

Header Propagation

Incoming message headers are not propagated to the target message by default. To propagate a header, you must define an expression that sets the header you want on the target message.

Example: Propagating a Source Header to the Target

This example propagates the my-header header from the source message to the target message:

solace:
   connector:
      workflows:
         0:
           transform:
             expressions:
               - transform: "target['headers']['my-header'] = source['headers']['my-header']"

Payload Propagation

When no expressions reference target['payload'], the payload is copied from the source message to the target message.

Payload propagation always adheres to the source-payload.content-type and target-payload.content-type settings. If the source-payload.content-type and target-payload.content-type are both set to application/json, the payload is interpreted and serialized as JSON, even though no transformation expressions are defined. See Payload Content Types for details.

If you define at least one expression to set the target['payload'] or any element within the target payload, the payload is not copied from the source message to the target message, so you must build the target payload from scratch.

Example: Setting a Static Value

This example sets the city element in the JSON target payload to Toronto:

solace:
  connector:
    workflows:
      0:
        transform:
          source-payload:
            content-type: application/json
          target-payload:
            content-type: application/json
          expressions:
            - transform: "target['payload']['city'] = 'Toronto'"

The resultant target payload will be exactly {"city":"Toronto"} regardless of what was in the source payload.

Example: Copying the Payload and Changing a Value

This example copies the JSON source payload to the JSON target payload, then sets the office element in the target payload to HQ:

solace:
  connector:
    workflows:
      0:
        transform:
            source-payload:
              content-type: application/json
            target-payload:
              content-type: application/json
            expressions:
              # Copy the payload from the source to the target
              - transform: "target['payload'] = source['payload']"
              # Set the `office` element in the target payload to `HQ`
              - transform: "target['payload']['office'] = 'HQ'"

If the target field doesn't exist, the transformation adds it. For example, if the source payload is {"a": "b"}, then the target payload resulting from the two transforms above is {"a": "b", "office": "HQ"}. As this example implies, you can also use this strategy to copy the source payload to the target payload and then modify parts of the target payload as needed.

Saving Intermediate Values

You can save intermediate values during the transformation process by using the var variable.

By saving values in the var variable, you avoid the need to recompute the same value or write complex expressions multiple times.

Example: Saving a Function Result

This example saves the return value from splitting the my-header source header to the split-result variable:

solace:
  connector:
    workflows:
      0:
        transform:
          expressions:
            # Split the `my-header` source header by `,` and save the result to the `split-result` variable
            - transform: "var['split-result'] = #splitString(source['headers']['my-header'], ',', 3)"
            # Set the `result-0` header to the first element of the split result
            - transform: "target['headers']['result-0'] = var['split-result'][0]"
            # Set the `result-1` header to the second element of the split result
            - transform: "target['headers']['result-1'] = var['split-result'][1]"
            # Set the `result-2` header to the third element of the split result
            - transform: "target['headers']['result-2'] = var['split-result'][2]"

Mapping and Transformation Example

Let's look at a transformation example. Consider a JSON message payload with the following structure:

{
    "airline": "ExampleAirline",
    "region": "Ontario",
    "requestId": 44334,
    "flight": {
        "flightModel": "boeing737",
        "flightRoute": "international"
    },
    "origin": "yow",
    "destination": "ewr",
    "status": "boarding",
    "passengers": "300,250,10",
    "lastUpdated": "2024-01-05T14:30:00"
}

To process this message, we'll create a transformation that:

1. Builds a dynamic routing header using the flight details

2. Converts the comma-separated passenger field into a structured format

Here's the transformation configuration:

solace:
  connector:
    workflows:
      0:
        transform:
          source-payload:
            content-type: application/json
          target-payload:
            content-type: application/json
          expressions:
            # Set the dynamic destination header to the value of `{airline}/{destination}/{origin}` from the source payload
            - transform: "target['headers']['scst_targetDestination'] = #joinString('/', source['payload']['airline'], source['payload']['destination'], source['payload']['origin'])"

            # Restructure the 'passengers' field from a comma-separated string into an object with three distinct fields
            - transform: "target['payload'] = source['payload']"
            - transform: "var['passengerDistribution'] = #splitString(source['payload']['passengers'], ',', 3)"
            - transform: "target['payload']['passengers'] = {:}" # Overwrite the 'passengers' field with an empty object
            - transform: "target['payload']['passengers']['capacity'] = #convertStringToNumber(var['passengerDistribution'][0])"
            - transform: "target['payload']['passengers']['occupied'] = #convertStringToNumber(var['passengerDistribution'][1])"
            - transform: "target['payload']['passengers']['personnel'] = #convertStringToNumber(var['passengerDistribution'][2])"

After applying this transformation, the output message header would be:

scst_targetDestination: ExampleAirline/ewr/yow

The output message payload in JSON format would be:

{
    "airline": "ExampleAirline",
    "region": "Ontario",
    "requestId": 44334,
    "flight": {
        "flightModel": "boeing737",
        "flightRoute": "international"
    },
    "origin": "yow",
    "destination": "ewr",
    "status": "boarding",
    "passengers": {
        "capacity": 300,
        "occupied": 250,
        "personnel": 10
    },
    "lastUpdated": "2024-01-05T14:30:00"
}