Advanced SERDES Configuration and Cross-Protocol Messaging

This page covers advanced SERDES Collection configuration options and cross-protocol messaging capabilities. For getting started information and basic configuration, see Serialization and Deserialization with Solace Schema Registry.

The following topics provide detailed information about specialized configurations and cross-protocol message translation:

• Advanced Configuration
• Cross-Protocol Message Translation

Advanced Configuration

The following configurations support specialized use cases and advanced scenarios. Most applications won't need these settings, but they provide fine-grained control over SERDES behavior when needed. These configuration options allow you to control automatic schema registration, customize header formats for cross-protocol compatibility, and configure format-specific encoding behaviors, which is useful for streamlining development workflows, optimizing message translation between protocols, and handling complex schema structures in Avro and JSON Schema applications.

• Auto-Registration Configuration
• SERDES Header Configuration
• Avro-Specific Configuration
• JSON Schema-Specific Configuration

Auto-Registration Configuration

Auto-registration allows serializers to automatically register schemas in Solace Schema Registry when they don't exist during serialization operations. This feature simplifies schema management by eliminating the need to manually register schemas before using them in your applications. When auto-registration is enabled, the serializer will attempt to register the schema if it cannot find a matching schema in the registry for the given artifact reference.

Auto-registration is particularly useful in development environments where schemas are frequently updated, or in scenarios where you want to streamline the deployment process by allowing applications to self-register their schemas. However, in production environments, you may want to disable auto-registration to maintain strict control over schema evolution and prevent unauthorized schema modifications.

• solace.registry.auto-register-artifact—A boolean flag that controls whether schemas should be automatically registered when they don't exist in the registry during serialization. When set to true, the serializer will attempt to register the schema if it's not found in the registry. When set to false (the default), the serializer will throw an exception if the schema is not found. This property only affects serialization operations.
• solace.registry.auto-register-artifact-if-exists—Controls the behavior when auto-registering artifacts that might already exist in the registry. This property works in conjunction with solace.registry.auto-register-artifact and only takes effect when auto-registration is enabled. The available options are:
  • IfArtifactExists.CREATE_VERSION—If a schema with the same content already exists in the registry, create a new version of that schema. This is useful when you want to maintain version history even for identical schemas.
  • IfArtifactExists.FAIL—If a schema with the same name already exists in the registry, the registration will fail and throw a SerializationException. This provides strict control over schema registration.
  • IfArtifactExists.FIND_OR_CREATE_VERSION—If a schema with the same content already exists in the registry, use the existing schema. Otherwise, create a new version. This is the default behavior and provides the most flexible approach to schema management.

For protocol-specific code examples showing how to configure these properties, see API and Protocol-Specific Implementation Guides. For JSON Schema-specific auto-registration requirements, see JSON Schema Auto-Registration.

SERDES Header Configuration

The solace.registry.schema-header-identifiers property allows you to select different SERDES header formats for schema identification in message headers. This configuration involves a tradeoff between efficiency and interoperability, enabling you to optimize your messaging system based on your specific requirements and integration scenarios.

When a message producer sends a message, the serializer component encodes the data according to a registered schema and includes a schema identifier in the message headers. The format of this schema identifier can be configured to balance performance optimization with cross-protocol compatibility needs.

The solace.registry.schema-header-identifiers property accepts values from the SchemaHeaderId enum, which provides two header format options:

• SchemaHeaderId.SCHEMA_ID(default)—Efficient long header configuration optimized for performance.
• SchemaHeaderId.SCHEMA_ID_STRING—String header configuration for optimal interoperability across protocols.

Efficiency vs. Interoperability Considerations

The choice between header formats represents a tradeoff:

• Efficiency—The default SCHEMA_ID option provides the best performance and efficiency by using a compact 8-byte long value for schema identification. This binary format minimizes header overhead and processing time, making it ideal for high-throughput Solace-to-Solace messaging scenarios.
• Interoperability—The SCHEMA_ID_STRING option ensures optimal compatibility between different messaging protocols by using human-readable string values. While this introduces some additional header processing overhead, it enables message translation and consumption across different messaging protocol environments. REST messaging serves as a specific example where string-only headers enable easier interoperability between protocols. When messages need to be consumed by REST endpoints or translated between different messaging systems, string-based schema identifiers provide better compatibility and easier debugging capabilities.

Configuration

To configure the solace.registry.schema-header-identifiers property:

• Set to SchemaHeaderId.SCHEMA_ID (the default) for maximum efficiency
• Set to SchemaHeaderId.SCHEMA_ID_STRING for optimal interoperability

For protocol-specific code examples, see API and Protocol-Specific Implementation Guides.

Avro-Specific Configuration

Apache Avro is a data serialization system that provides a compact binary data format and schema evolution capabilities. Avro serialization allows you to encode and decode complex data structures efficiently—encoding before publishing them as messages and decoding when consuming messages. This ensures consistent data formatting across your messaging applications and enables schema evolution as your application requirements change.

This section covers Avro-specific configuration properties. For common SERDES configuration properties, see Getting Started with SERDES.

All SERDES properties are set in a configuration map. For protocol-specific examples of how to create and configure this map, see API and Protocol-Specific Implementation Guides.

• Dereferencing Schemas
• Avro Encoding Types
• Avro Record Type Configuration

Dereferencing Schemas

The solace.avro.dereferenced-schema property controls how serializers handle Avro schemas that contain references to other schemas. In Avro schema design, there are two primary approaches to managing complex schemas:

• Referenced schemas—These schemas are structured to reference or include other shared schema components, enabling a modular and reusable design. Common types can be defined once and reused across multiple schemas. Referenced schemas are reusable and easy to maintain.
• Dereferenced schemas—These schemas, also called flat schemas, have all references expanded inline, creating a single, self-contained definitions with no external dependencies. Dereferenced schemas simplify consumption by eliminating the need to resolve multiple schema references at runtime.

When set to true (the default), the solace.avro.dereferenced-schema property tells the serializer to treat the schema in the record as fully dereferenced and use it directly for registry lookups. The serializer does not attempt to extract or manage referenced sub-schemas, which simplifies schema handling in your application.

This configuration is useful in scenarios where you want to:

• simplify schema handling in your application by avoiding schema reference resolution.
• optimize performance by reducing the number of schema registry lookups.

When solace.avro.dereferenced-schema is set to false, the serializer will treat the provided schema as one that needs to be decomposed into reference schemas. This allows applications to use one Avro schema file to represent multiple Avro schemas in the registry, which can reduce storage of common sub-schemas in the registry.

To configure a serializer to decompose schemas into references, set solace.avro.dereferenced-schema to false. For protocol-specific code examples, see API and Protocol-Specific Implementation Guides.

Avro Encoding Types

The solace.avro.encoding-type property allows you to specify the encoding format used by the AvroSerializer to convert data to bytes. Avro supports two primary encoding formats, each with different characteristics and use cases:

• AvroEncoding.BINARY—Uses Avro's DirectBinaryEncoder to produce a compact binary representation of the data. This is the default encoding and provides the most efficient serialization in terms of message size and processing performance.
• AvroEncoding.JSON—Uses Avro's JsonEncoder to produce a human-readable JSON representation of the data. While less efficient than binary encoding, this format is useful for debugging, logging, and interoperability with systems that expect JSON.

The encoding type affects only the wire format of the serialized data; it does not change how the schema is resolved or how the data is structured. Both encoding types maintain full compatibility with the Avro schema system.

The encoding type is set as an Avro SERDES message header. This impacts the AvroDeserializer, which reads the assigned header value and attempts to use the appropriate decoder. If no header is present, the AvroDeserializer assumes the message was encoded with binary and attempts to decode it accordingly.

To configure a serializer to use JSON encoding, set solace.avro.encoding-type to AvroEncoding.JSON. For protocol-specific code examples, see API and Protocol-Specific Implementation Guides.

When choosing an encoding type, consider these factors:

• Use BINARY encoding when optimizing for performance, bandwidth efficiency, or message throughput.
• Use JSON encoding when debugging applications or when human readability is important.

Avro Record Type Configuration

The solace.avro.record-type property specifies the type of Avro record representation to use during serialization and deserialization. Avro supports different record representations that offer varying levels of flexibility and performance:

• Specific records—Strongly-typed objects generated from Avro schemas at compile time. These provide type safety and better performance but require code generation.
• Generic records—Dynamic records that work with any schema at runtime without requiring pre-generated classes. These offer more flexibility but with slightly reduced performance.

The choice between specific and generic records depends on your application requirements. Use specific records when you have known schemas at development time and want type safety. Use generic records when working with dynamic schemas or when schema information is only available at runtime.

To configure the record type, set solace.avro.record-type to the appropriate value for your implementation. For protocol-specific code examples and available record type values, see API and Protocol-Specific Implementation Guides.

JSON Schema-Specific Configuration

JSON Schema is a vocabulary that defines how to annotate and validate JSON documents. JSON Schema serialization allows you to encode and decode structured JSON data efficiently—encoding before publishing them as messages and decoding when consuming messages. This ensures consistent data formatting across your messaging applications and enables schema evolution as your application requirements change.

This section covers JSON Schema-specific configuration properties. For common SERDES configuration properties, see Getting Started with SERDES.

All SERDES properties are set in a configuration map. For protocol-specific examples of how to create and configure this map, see API and Protocol-Specific Implementation Guides.

• JSON Schema Auto-Registration
• JSON Schema Validation
• Type Property Configuration

JSON Schema Auto-Registration

When using auto-registration with JSON Schema serialization, you must also specify the schema location using solace.registry.schema-location to provide the classpath resource path to the schema file. Unlike Avro schemas, which embed schema information in the serialized data, JSON Schema requires an explicit schema file reference for auto-registration to work properly.

To configure the schema location, set solace.registry.auto-register-artifact to true and specify the classpath resource path using solace.registry.schema-location (for example, "json-schema/user.json"). For protocol-specific code examples, see API and Protocol-Specific Implementation Guides.

JSON Schema Validation

The solace.jsonschema.validate-schema property controls whether JSON schema validation is performed during serialization and deserialization operations. When enabled, the serializer and deserializer will validate JSON data against its registered schema to ensure data conformity and catch schema violations early in the processing pipeline.

Schema validation provides several benefits:

• Data integrity—Ensures that JSON data conforms to the expected schema structure before processing.
• Early error detection—Catches schema violations during serialization/deserialization rather than downstream processing.
• Consistency enforcement—Guarantees that all messages follow the defined data contract.

However, enabling validation does introduce some performance overhead, so you may want to disable it in performance-critical scenarios where data integrity is guaranteed by other means.

The solace.jsonschema.validate-schema property accepts a boolean value:

• true (default)—Enables JSON schema validation during serialization and deserialization operations.
• false—Disables JSON schema validation, improving performance but removing data integrity checks.

To configure JSON schema validation, set solace.jsonschema.validate-schema to true (the default) to enable validation, or false to disable it for improved performance. For protocol-specific code examples, see API and Protocol-Specific Implementation Guides.

Type Property Configuration

The solace.jsonschema.type-property configuration specifies the JSON schema property name that contains the fully qualified type name for deserialization. The JsonSchemaDeserializer uses this property to determine the target class when converting JSON data back to strongly-typed objects, enabling proper type reconstruction during the deserialization process. This feature supports polymorphic deserialization across different programming languages, including Java and .NET.

This property is particularly important when:

• Your JSON schema includes type information for object reconstruction.
• You need to deserialize JSON data into specific strongly-typed classes rather than generic JSON objects.
• Your application uses polymorphic types that require runtime type resolution.

The solace.jsonschema.type-property accepts a string value that specifies the property name in your JSON schema:

• Default value varies by language: "javaType" for Java implementations, configurable for other languages (for example, .NET applications commonly use custom property names like "customDotnetType").
• Custom value: Any valid JSON property name that contains the fully qualified type name.

To configure the type property, set solace.jsonschema.type-property to the desired property name. The format of the type name depends on the programming language—Java uses fully qualified class paths (for example, "com.example.User"), while .NET uses fully qualified type names (for example, "MyApp.Models.User, MyApp"). For protocol-specific and language-specific code examples, see API and Protocol-Specific Implementation Guides.

The following example shows a JSON schema that includes a javaType property. When the JsonSchemaDeserializer processes JSON data conforming to this schema, it will use the type value to instantiate the correct class during deserialization:

{
  "$schema": "https://json-schema.org/draft-07/schema",
  "type": "object",
  "javaType": "com.example.User",
  "properties": {
    "name": {
      "type": "string"
    },
    "email": {
      "type": "string",
      "format": "email"
    },
    "age": {
      "type": "integer",
      "minimum": 0
    }
  },
  "required": ["name", "email"]
}

Cross-Protocol Message Translation

SERDES supports cross-protocol messaging, allowing messages to be published using one protocol and consumed using another. The Solace broker performs automatic message translation and header mapping to preserve SERDES schema information across protocols.

Message Type Translation Rules

The broker translates messages between protocols according to the following rules. For detailed information on protocol metadata and payload encoding interactions, see Protocol Metadata and Payload Encoding Interactions:

• AMQP to REST Translation—AMQP application properties are mapped to HTTP headers with the solace-user-property- prefix, preserving SERDES schema information for REST consumers.
• AMQP to Solace Message Format (SMF) Translation—AMQP application properties are mapped to Solace Message Format (SMF) user properties, allowing Solace Messaging API consumers to access SERDES headers.
• REST to Solace Message Format (SMF) Translation—HTTP headers with the solace-user-property- prefix are mapped to SMF user properties, preserving SERDES schema information for Solace Messaging API consumers.
• Message Payload Preservation—Message payloads are preserved during protocol translation, ensuring that serialized data remains intact regardless of the consuming protocol. Note that during conversion, the event broker may wrap the payload in additional bytes as required by the target protocol's message format.

SERDES Compatibility with Message Types

Different protocols support different message payload types for SERDES operations:

• Binary Payload Support (Binary Attachment - Unstructured Bytes)—All protocols support binary message payloads (BytesMessage in JCSMP, application/octet-stream in REST, BytesMessage in AMQP), where the SERDES deserializer processes the byte array directly. This is the recommended approach for optimal performance and compatibility.
• Text Payload Support (Binary Attachment - Text)—Some protocols support text payloads for deserialization (TextMessage in JCSMP, text-based Content-Types in REST, TextMessage in AMQP), where the deserializer converts the UTF-8 text content to bytes before processing. Text payloads are not supported for serialization. This enables flexibility in message handling and cross-protocol scenarios.

This cross-protocol compatibility allows applications using different protocols to work seamlessly together without requiring changes to the application code. Schema validation and message deserialization work consistently across all supported protocols.

Protocol-Specific Header Mapping

SERDES headers are transmitted differently depending on the messaging protocol. Understanding these mappings is critical for cross-protocol messaging:

Protocol	Header Location	Notes
REST (HTTP)	HTTP headers with solace-user-property- prefix	Example: solace-user-property-solace.registry.schema.id Use SCHEMA_ID_STRING format for optimal REST compatibility All values transmitted as strings in HTTP
AMQP 1.0	AMQP application properties	Native AMQP 1.0 protocol feature Supports typed values (strings, integers, etc.) Client library specific: Apache Qpid JMS requires setValidatePropertyNames(false) for dots/hyphens AMQPNetLite: Access via Message.ApplicationProperties
Solace Message Format (SMF) / Solace Messaging API	SMF user properties	Native to Solace messaging APIs Automatically mapped from AMQP/REST by broker

Cross-Protocol Message Flow Examples

The following examples demonstrate how SERDES headers and payloads flow between different protocols. The broker automatically handles protocol translation and header mapping.

Flow 1: REST Publisher → SMF Consumer

1. Publisher Side (REST)—The REST client serializes the message payload using the appropriate SERDES serializer, which populates a headers map. The application adds these headers to the HTTP request with the solace-user-property- prefix.
2. Message Transmission—The serialized payload and SERDES headers are transmitted via HTTP POST to the event broker REST endpoint.
3. Broker Translation—The broker extracts HTTP headers with solace-user-property- prefix and maps them to SMF user properties.
4. Consumer Side (SMF)—The Solace Messaging API application receives the message, extracts the SERDES headers from SMF user properties, and uses SerdeMessage.deserialize() to reconstruct the payload.

Flow 2: SMF Publisher → REST Consumer

1. Publisher Side (SMF)—The Solace Messaging API application serializes the message payload using SerdeMessage.serialize(), which automatically adds schema identification headers as SMF user properties.
2. Message Transmission—The serialized payload and SERDES headers are published to the event broker using Solace Messaging API.
3. Broker Translation—The broker maps SMF user properties to HTTP headers with solace-user-property- prefix.
4. Consumer Side (REST)—The REST consumer application receives the HTTP request via a REST Delivery Point (RDP), extracts SERDES headers from HTTP headers, and uses the appropriate deserializer to reconstruct the payload.

Flow 3: AMQP Publisher → REST Consumer

1. Publisher Side (AMQP)—The AMQP client serializes the message payload using the appropriate SERDES serializer, which populates a headers map. The application sets these as AMQP application properties on the message.
2. Message Transmission—The serialized payload and SERDES headers are transmitted via AMQP 1.0 to the event broker.
3. Broker Translation—The broker maps AMQP application properties to HTTP headers with solace-user-property- prefix.
4. Consumer Side (REST)—The REST consumer application receives the HTTP request via a REST Delivery Point (RDP), extracts SERDES headers from HTTP headers, and uses the appropriate deserializer to reconstruct the payload.

Flow 4: REST Publisher → AMQP Consumer

1. Publisher Side (REST)—The REST client serializes the message payload using the appropriate SERDES serializer and adds headers to the HTTP request with the solace-user-property- prefix.
2. Message Transmission—The serialized payload and SERDES headers are transmitted via HTTP POST to the event broker REST endpoint.
3. Broker Translation—The broker maps HTTP headers with solace-user-property- prefix to AMQP application properties.
4. Consumer Side (AMQP)—The AMQP consumer receives the message, extracts SERDES headers from AMQP application properties, retrieves the message payload bytes, and uses the appropriate deserializer to reconstruct the payload.

Flow 5: AMQP Publisher → SMF Consumer

1. Publisher Side (AMQP)—The AMQP client serializes the message payload using the appropriate SERDES serializer and sets SERDES headers as AMQP application properties.
2. Message Transmission—The serialized payload and SERDES headers are transmitted via AMQP 1.0 to the event broker.
3. Broker Translation—The broker maps AMQP application properties to SMF user properties.
4. Consumer Side (SMF)—The Solace Messaging API application receives the message, extracts SERDES headers from SMF user properties, and uses SerdeMessage.deserialize() to reconstruct the payload.

Flow 6: SMF Publisher → AMQP Consumer

1. Publisher Side (SMF)—The Solace Messaging API application serializes the message payload using SerdeMessage.serialize(), which automatically adds schema identification headers as SMF user properties.
2. Message Transmission—The serialized payload and SERDES headers are published to the event broker.
3. Broker Translation—The broker maps SMF user properties to AMQP application properties.
4. Consumer Side (AMQP)—The AMQP consumer receives the message, extracts SERDES headers from AMQP application properties, and uses the appropriate deserializer to reconstruct the payload.