Serializing and Deserializing Messages with the PubSub+ JCSMP API

You can use the PubSub+ JCSMP API with the SERDES Collection to handle structured message payloads efficiently. SERDES, which stands for Serialization and Deserialization, allows applications to convert complex data structures into a compact, transmittable format when publishing messages and to reconstruct them when consuming messages. This is particularly useful when integrating with schema registries. The SERDES Collection refers to the set of serializer and deserializer libraries, such as Solace Avro SERDES for Java, Solace JSON Schema SERDES for Java, and Generic SERDES for Java. You can use these libraries to integrate applications with Solace Schema Registry for schema-based message serialization and deserialization.

• Using the SERDES Collection with Solace Schema Registry
• Deploying with Maven
• Common Serializer and Deserializer Configuration
• Avro-Specific Serializer and Deserializer Configuration
• JSON Schema-Specific Serializer and Deserializer Configuration
• Serializing and Deserializing Messages with Avro
• Serializing and Deserializing Messages with JSON Schema

Using the SERDES Collection with Solace Schema Registry

In modern messaging systems, managing data formats across distributed applications can present challenges. As you scale your applications, differences in how message producers and consumers serialize and interpret data can lead to issues with data consistency and version compatibility. Without a shared understanding of the message format, mismatches such as missing fields, unexpected data types, or incompatible versions can result in runtime errors. To prevent these problems and streamline data exchange in Solace messaging environments, we recommend you use Solace Schema Registry with the SERDES Collection.

What is a schema?

A schema defines the structure, data types, and validation rules for messages exchanged between producers and consumers, ensuring that they agree on the format of the data being exchanged. In formats like Avro or JSON Schema, a schema explicitly describes each field’s name, type, and if it is required. This enables compatibility checks, validation, and tooling support, and ensures consistent data interpretation across services, especially when schemas are managed in a schema registry.

What is a schema registry?

A schema registry is a centralized service that stores and manages schemas used for serializing and deserializing structured data. It ensures that applications exchanging messages follow a consistent data format, which prevents compatibility issues between message producers and consumers. By using Solace Schema Registry, you can enforce data validation, track schema versions, and support schema evolution in a way that maintains compatibility with applications using older schema versions.

How does everything work together?

When integrated with the SERDES Collection, your applications can automatically retrieve and apply the correct schema from Solace Schema Registry during message serialization and deserialization. When a message producer sends a message, the serializer component encodes the data according to a registered schema, including a schema identifier stored in the SERDES header appended to your message. On the message consumer side, the API extracts the schema identifier from the header, retrieves the corresponding schema from the registry, and ensures the message is correctly parsed. This header-based approach allows for efficient schema resolution without modifying the message payload itself. This interaction enables dynamic schema resolution, eliminates the need to hardcode schemas in applications, and simplifies the management of structured data formats like Avro and JSON Schema in your messaging applications.

Steps for Using the SERDES Collection with Solace Schema Registry

1. Set up a Solace Schema Registry instance then define and register your schemas.
2. Choose a serialization format. Solace currently supports Avro and JSON Schema.
3. Configure your SERDES objects (Avro or JSON Schema).
   • Both JSON and Avro share a common SERDES configuration API for registry access, resolution, and caching.
4. Use a serializer to serialize outbound messages and a deserializer to deserialize inbound messages.

For more information about Solace schema registries, see Solace Schema Registry.

Deploying with Maven

To use the SERDES Collection with the PubSub+ JCSMP API, you need to include the appropriate Maven dependencies in your project. The SERDES libraries are distributed as separate artifacts that you can include based on your serialization format requirements.

The following sections show you how to configure your Maven project to include the necessary dependencies for schema registry integration:

• SERDES BOM Dependencies (Recommended)
• Avro Serializer Dependencies
• JSON Schema Serializer Dependencies
• Generic Serializer Dependencies

SERDES BOM Dependencies (Recommended)

The SERDES Bill of Materials (BOM) provides a centralized way to manage compatible versions of all Solace Schema Registry SERDES components. Using the BOM ensures that all SERDES dependencies are aligned with tested and compatible versions. This simplifies dependency management and reduces version conflicts in your applications.

The BOM includes version management for:

• Avro SERDES components
• JSON Schema SERDES components
• Common SERDES libraries
• Compatible versions of underlying dependencies

To use the SERDES BOM, add it to your Maven pom.xml file in the dependencyManagement section:

<dependencyManagement>
    <dependencies>
        <dependency>
            <groupId>com.solace</groupId>
            <artifactId>solace-schema-registry-serdes-bom</artifactId>
            <version>1.0.0</version>
            <type>pom</type>
            <scope>import</scope>
        </dependency>
    </dependencies>
</dependencyManagement>

After importing the BOM, you can add SERDES dependencies without specifying versions:

<dependencies>
    <!-- JCSMP API -->
    <dependency>
        <groupId>com.solacesystems</groupId>
        <artifactId>sol-jcsmp</artifactId>
        <version>10.28.0</version>
    </dependency>
    
    <!-- Avro SERDES (version managed by BOM) -->
    <dependency>
        <groupId>com.solace</groupId>
        <artifactId>solace-schema-registry-avro-serde</artifactId>
    </dependency>
    
    <!-- JSON Schema SERDES (version managed by BOM) -->
    <dependency>
        <groupId>com.solace</groupId>
        <artifactId>solace-schema-registry-jsonschema-serde</artifactId>
    </dependency>
</dependencies>

Avro Serializer Dependencies

The Avro serializer provides a specific implementation for Apache Avro serialization and deserialization. This dependency includes the AvroSerializer, AvroDeserializer, and Avro-specific configuration properties, and also brings in the required common SERDES components used across all schema formats.

Add the following dependency to your Maven pom.xml file:

<dependencies>
    <dependency>
        <groupId>com.solace</groupId>
        <artifactId>solace-schema-registry-avro-serde</artifactId>
        <version>1.0.0</version>
    </dependency>
</dependencies>

For the latest version information and additional details about the Avro SERDES artifact, see the Maven Central Repository.

JSON Schema Serializer Dependencies

The JSON Schema serializer provides a specific implementation for JSON Schema serialization and deserialization. This dependency includes the JsonSchemaSerializer, JsonSchemaDeserializer, and JSON Schema-specific configuration properties, and also brings in the required common SERDES components used across all schema formats.

Add the following dependency to your Maven pom.xml file:

<dependencies>
    <dependency>
        <groupId>com.solace</groupId>
        <artifactId>solace-schema-registry-jsonschema-serde</artifactId>
        <version>1.0.0</version>
    </dependency>
</dependencies>

For the latest version information and additional details about the JSON Schema SERDES artifact, see the Maven Central Repository.

Generic Serializer Dependencies

The Generic SERDES for Java provides simple string serialization and deserialization capabilities with no Solace Schema Registry interaction. Generic SERDES for Java is automatically included when you update to version 10.28 or later of the PubSub+ JCSMP API.

Add the following dependencies to your Maven pom.xml file:

<dependencies>
  <dependency>
    <groupId>com.solacesystems</groupId>
    <artifactId>sol-jcsmp</artifactId>
    <version>10.28.0</version>  
  </dependency>
</dependencies>

Common Serializer and Deserializer Configuration

The Common SERDES Configuration section provides the foundation for all serialization and deserialization operations in the SERDES Collection, regardless of the specific format being used, such as Avro or JSON Schema. This section covers the common configuration options that enable your applications to interact with Solace Schema Registry, manage schema resolution, and implement schema selection strategies.

The following Java imports are required for implementing serialization and deserialization using the SERDES Collection:

import com.solace.serdes.Serializer;
import com.solace.serdes.Deserializer;
import com.solace.serdes.common.resolver.config.SchemaResolverProperties;
import java.util.HashMap;
import java.util.Map;

All SERDES properties are set in a HashMap object with the put() method:

Map<String, Object> config = new HashMap<>();
// set all your configuration parameters with config.put()

• Authentication
• Artifact Resolver Strategies (Serialization Only)
• Schema Resolution Options
• Auto-Registration Configuration
• SERDES Header Configuration

Authentication

The Generic SERDES for Java supports multiple authentication methods for connecting to Solace Schema Registry. This section covers how to configure authentication credentials for your SERDES objects, including basic username and password authentication, and how to implement them in both secure and non-secure connection contexts. You specify the authentication settings when you configure your schema registry client, which allows you to control access permissions and maintain the security of your schemas.

The following properties configure how your application connects to Solace Schema Registry:

• SchemaResolverProperties.REGISTRY_URL—The URL of Solace Schema Registry where schemas are stored. This property is required in all serializer and deserializer configurations. The registry URL must point to the Registry REST API endpoint. The default path for this endpoint is /apis/registry/v3.
• SchemaResolverProperties.AUTH_USERNAME—The username to use to authenticate with Solace Schema Registry.
• SchemaResolverProperties.AUTH_PASSWORD—The password to use to authenticate with Solace Schema Registry.
• SchemaResolverProperties.TRUST_STORE_PATH—The file path to the truststore containing SSL/TLS certificates for secure connections.
• SchemaResolverProperties.TRUST_STORE_PASSWORD—The password required to access the truststore file.
• SchemaResolverProperties.VALIDATE_CERTIFICATE—A flag that determines whether SSL certificate validation is enabled (true) or disabled (false).
  

  We recommend that you never disable VALIDATE_CERTIFICATE in production environments because it creates a security vulnerability.

The following code snippets show you how to set these properties based on which authentication option you use:

• Authentication, plain text connection—Used in restricted environments where network access is controlled but authentication is still required.

  config.put(SchemaResolverProperties.REGISTRY_URL, "http://localhost:8081/apis/registry/v3");
  
  // Set authentication credentials
  config.put(SchemaResolverProperties.AUTH_USERNAME, "myUsername");
  config.put(SchemaResolverProperties.AUTH_PASSWORD, "myPassword");

• Authentication, secure connection—The recommended production setup. This setup ensures both data encryption and access control for secure schema management.

  // Use HTTPS and non-localhost hostname for secure communication with the schema registry
  config.put(SchemaResolverProperties.REGISTRY_URL, "https://{your_hostname}:443/apis/registry/v3");
  
  // Authentication credentials
  config.put(SchemaResolverProperties.AUTH_USERNAME, "myUsername");
  config.put(SchemaResolverProperties.AUTH_PASSWORD, "myPassword");
  
  // Configure TLS properties for secure connection, this includes the truststore path and password
  config.put(SchemaResolverProperties.TRUST_STORE_PATH, "path/to/truststore");
  config.put(SchemaResolverProperties.TRUST_STORE_PASSWORD, "truststore_password");
  
  config.put(SchemaResolverProperties.VALIDATE_CERTIFICATE, true);

• Advanced authentication—Used when you configure Schema Registry with an external identity provider such as Microsoft Entra ID.

  config.put(SchemaResolverProperties.REGISTRY_URL, "https://{your_hostname}:443/apis/registry/v3");
  
  // For OAuth authentication, set your client ID as AUTH_USERNAME and your client secret as AUTH_PASSWORD:
  config.put(SchemaResolverProperties.AUTH_USERNAME, "your-client-id");
  config.put(SchemaResolverProperties.AUTH_PASSWORD, "your-client-secret");

  When using advanced authentication flows, you configure the same AUTH_USERNAME and AUTH_PASSWORD properties. In this case, AUTH_USERNAME maps to your OAuth Client ID, and AUTH_PASSWORD maps to your OAuth Client Secret from your OAuth configuration. For more information about configuring external identity providers with Schema Registry, see Deploying and Configuring Solace Schema Registry with Docker or Podman.

Artifact Resolver Strategies (Serialization Only)

Artifact resolver strategies define how a schema or artifact is dynamically resolved at runtime for serialization operations performed by the SERDES Collection. These strategies determine which schema to fetch from Solace Schema Registry based on metadata like Solace topics, destination IDs, and record IDs. Artifact resolver strategies are only applied during serialization, because the serializer is responsible for selecting or creating the artifact ID to register or reference a schema. Deserializers simply use the artifact ID embedded in the incoming message to retrieve the correct schema, without needing to determine how that ID was formed.

Some important terms:

• record—A structured data object, for example an Avro or JSON Schema message, that is serialized or deserialized using a schema from Solace Schema Registry.
• artifact—A named and versioned container in Solace Schema Registry that holds one or more schema versions. Each artifact is identified by an artifactId and optionally grouped using a groupId.
• ArtifactReference—A pointer to an existing artifact in Solace Schema Registry. When the artifact contains a schema, this reference can be used to locate and apply it during serialization or deserialization.
• artifactId—A unique identifier for an artifact in Solace Schema Registry, which typically contains a single schema.
• groupId—A logical grouping mechanism for artifacts in Solace Schema Registry. It allows organizing related schemas, for example all schemas for a specific application. If no groupId is specified, the value is default.

The available strategies for retrieving schemas are available:

• Destination ID Strategy
• Solace Topic ID Strategy
• Solace Topic ID Strategy with Profiles
• Record ID Artifact Resolver Strategy (Avro format only)

Destination ID Strategy

The DestinationIdStrategy automatically determines which schema to use during serialization. It extracts the destination name from the record metadata to use as the artifactId, while applying a default value as the groupId to construct the complete ArtifactReference. This approach eliminates the need for explicit schema specification on a per message basis because it creates a direct mapping between message destinations and schema artifacts stored in the registry. For successful implementation, Solace Schema Registry must contain schemas with artifactIds that correspond to the destination names used in your application's messaging infrastructure. The strategy is valuable in systems where different message types are routed to different destinations, as it reduces configuration overhead and decouples serialization logic from schema-specific details.

In summary, the DestinationIdStrategy:

• Uses the destination name defined by the parameter passed into the serialize() method call. In REST implementations, this is typically an endpoint name or path specified directly by the user. In PubSub+ JCSMP API implementations, this destination is automatically extracted from the message topic via the serialize() method.
• Uses this destination name as the artifactId in the ArtifactReference.
• Uses a default value as the groupId in the ArtifactReference.
• Uses the constructed ArtifactReference to locate the correct schema in Solace Schema Registry.

Here's how to configure a serializer to use the DestinationIdStrategy:

config.put(SchemaResolverProperties.ARTIFACT_RESOLVER_STRATEGY, DestinationIdStrategy.class);

Solace Topic ID Strategy

The SolaceTopicIdStrategy automatically determines which schema to use during serialization by mapping the destination name directly to the ArtifactReference, setting the destination string as the artifactId. This approach simplifies schema resolution by creating a direct correlation between Solace topic destinations and schema artifacts stored in the registry. For successful implementation, Solace Schema Registry must contain schemas with artifactIds that correspond to the topic names used in your Solace messaging infrastructure.

In summary, the SolaceTopicIdStrategy:

• Uses the destination name defined by the parameter passed into the serialize() method call. In REST implementations, this is typically an endpoint name or path specified directly by the user. In PubSub+ JCSMP API implementations, this destination is automatically extracted from the message topic via the serialize() method.
• Maps this destination name directly as the artifactId in the ArtifactReference.
• Uses the constructed ArtifactReference to locate the correct schema in Solace Schema Registry.

Here's how to configure a serializer to use the SolaceTopicIdStrategy without a profile:

config.put(SchemaResolverProperties.ARTIFACT_RESOLVER_STRATEGY, SolaceTopicIdStrategy.class);	

Solace Topic ID Strategy with Profiles

For more advanced mapping scenarios, the SolaceTopicIdStrategy can be used with a SolaceTopicProfile to provide flexible topic-to-schema mappings. This approach allows for wildcard topic expressions and custom artifact references, giving you greater control over schema resolution.

Using profiles with SolaceTopicIdStrategy provides several benefits:

• Support for wildcard topic expressions to match multiple topics with a single mapping. You can use * for single-level wildcards and > for multi-level wildcards. For more information about wildcards, see Wildcard Characters in Topic Subscriptions.
• The ability to map different topic patterns to specific schemas.
• Control over groupId, artifactId, and versioning for schema selection

The SolaceTopicProfile can be configured with different types of mappings:

Topic Expression Only Mapping

This method maps a Solace topic expression directly to an artifactId that matches the topic expression itself. The groupId is left as default. Use this method when your schema registry follows a convention where schemas are registered with an artifactId that matches your topic hierarchy.

The following code snippet creates three topic mapping patterns, exact, single-level wildcard, and multi-level wildcard, and adds them to a SolaceTopicProfile container:

// Create a new instance of SolaceTopicProfile
SolaceTopicProfile profile = new SolaceTopicProfile();

// Create mappings with topic expressions
SolaceTopicArtifactMapping mapping1 = SolaceTopicArtifactMapping.create("solace/samples");
SolaceTopicArtifactMapping mapping2 = SolaceTopicArtifactMapping.create("solace/*/sample");
SolaceTopicArtifactMapping mapping3 = SolaceTopicArtifactMapping.create("solace/>");

// Add mappings to profile
profile.add(mapping1);
profile.add(mapping2);
profile.add(mapping3);

// Add the profile to your serializer configuration, enabling the SolaceTopicIdStrategy to use your topic-to-schema mapping
config.put(SchemaResolverProperties.STRATEGY_TOPIC_PROFILE, profile);

Topic Expression with Custom Artifact ID

This method maps a Solace topic expression to a specific, custom artifactId. The groupId is left as "default". Use this method when multiple topics should use the same schema, but your topic names do not match your schema registry naming conventions.

The following code snippet creates three topic-to-schema mappings that associate different Solace topic patterns with specific custom artifact IDs (User, NewUser, and OldUser) and adds them to a SolaceTopicProfile container:

// Create a new instance of SolaceTopicProfile
SolaceTopicProfile profile = new SolaceTopicProfile();

// Create mappings with topic expressions and custom artifact IDs
SolaceTopicArtifactMapping mapping1 = SolaceTopicArtifactMapping.create("solace/samples", "User");
SolaceTopicArtifactMapping mapping2 = SolaceTopicArtifactMapping.create("solace/*/sample", "NewUser");
SolaceTopicArtifactMapping mapping3 = SolaceTopicArtifactMapping.create("solace/>", "OldUser");

// Add mappings to profile
profile.add(mapping1);
profile.add(mapping2);
profile.add(mapping3);

// Add the profile to your serializer configuration, enabling the SolaceTopicIdStrategy to use your topic-to-schema mapping
config.put(SchemaResolverProperties.STRATEGY_TOPIC_PROFILE, profile);	

Topic Expression with Full ArtifactReference

This method maps a Solace topic expression to a complete ArtifactReference with a custom groupId, artifactId, and optional version for exact schema selection. Use this method when you need complete control over schema resolution, especially when you have multiple schema groups or when specific versions must be used.

The following code snippet creates two artifact references, one with version specification and one without, and maps them to specific topic patterns:

// Create a new instance of SolaceTopicProfile
SolaceTopicProfile profile = new SolaceTopicProfile();

// Create artifact reference
ArtifactReferenceBuilder builder = new ArtifactReferenceBuilder();
ArtifactReference reference = builder
        .groupId("com.solace.samples.serdes.avro.schema")
        .artifactId("User")
        .build();

// Create mapping with topic expression and artifact reference
SolaceTopicArtifactMapping mapping = SolaceTopicArtifactMapping.create("solace/samples", reference);
// Create mapping with version-specific reference
ArtifactReference versionedReference = builder
        .groupId("com.solace.samples.serdes.avro.schema")
        .artifactId("User")
        .version("0.0.1")
        .build();

SolaceTopicArtifactMapping versionedMapping = SolaceTopicArtifactMapping.create("solace/>", versionedReference);
// Add mappings to profile
profile.add(mapping);
profile.add(versionedMapping);

// Add the profile to your serializer configuration, enabling the SolaceTopicIdStrategy to use your topic-to-schema mapping
config.put(SchemaResolverProperties.STRATEGY_TOPIC_PROFILE, profile);

Schema Resolution Options

Schema resolution determines how your application retrieves, caches, and manages schemas when it interacts with Solace Schema Registry. This section covers configuration options, including caching strategies, and schema lookup options.

• Schema Caching Options
• Schema Lookup Options

Schema Caching Options

To reduce the overhead of repeated schema registry lookups, clients using the SERDES Collection can configure schema caching options. These settings control how long schemas are stored locally and how different types of lookups interact with the schema registry cache.

• SchemaResolverProperties.CACHE_TTL_MS—Determines how long schema artifacts remain valid in the cache before they need to be fetched again from the registry on the next relevant lookup. The default value is 30000ms or 30 seconds. Longer TTLs improve performance by reducing registry calls but risk using outdated schemas, shorter TTLs ensure fresher schemas but increase registry load. A zero TTL disables caching entirely, requiring registry fetches for every request. You can configure this property in several ways:

  // Example 1: Set cache TTL using a Long value
  config.put(SchemaResolverProperties.CACHE_TTL_MS, 5000L);
  
  // Example 2: Set cache TTL using a String value
  config.put(SchemaResolverProperties.CACHE_TTL_MS, "5000");
  
  // Example 3: Set cache TTL using a Duration object
  config.put(SchemaResolverProperties.CACHE_TTL_MS, Duration.ofSeconds(5));
  
  // Example 4: Disable caching completely
  config.put(SchemaResolverProperties.CACHE_TTL_MS, 0L);

• SchemaResolverProperties.USE_CACHED_ON_ERROR—Controls whether to use cached schemas when schema registry lookup errors occur. When enabled, schema resolution uses cached schemas instead of throwing exceptions after retry attempts are exhausted, improving resilience during registry outages. When disabled, the default, the API throws an exception when registry lookup errors occur. The following code snippet shows how to configure this property:

  // Use cached schemas when registry lookups fail (resilient mode)
  config.put(SchemaResolverProperties.USE_CACHED_ON_ERROR, true);
  
  // Throw exceptions when registry lookups fail (strict mode, default value)
  config.put(SchemaResolverProperties.USE_CACHED_ON_ERROR, false);

• SchemaResolverProperties.CACHE_LATEST(Serializers Only)—Controls whether schema lookups that specify latest or do not include an explicit version (no-version) create additional cache entries, so that future "latest" or versionless lookups can be resolved using the cache instead of querying the registry again. When enabled, the default behavior allows latest and versionless (no-version) schema lookups to create additional cache entries, enabling subsequent latest or no-version lookups to use the cached schema without contacting the registry. When you disable this property, only the resolved version is cached, meaning every latest or versionless lookup must query the registry to determine the current latest version.

  // Example 1: Enable caching of 'latest' version lookups (default behavior)
  config.put(SchemaResolverProperties.CACHE_LATEST, true);
  
  // Example 2: Disable caching of 'latest' version lookups
  // When disabled, only the resolved version is cached, requiring subsequent
  // latest/no-version lookups to be fetched from the registry
  config.put(SchemaResolverProperties.CACHE_LATEST, false);

  • When an explicit schema version is specified, the CACHE_LATEST setting has no effect on schema lookup results.
  • CACHE_LATEST only affects caching for serialization operations. It does not apply to schema references, which are always resolved through direct registry lookups, bypassing the cache.

Schema Lookup Options

When you use the SERDES Collection with Solace Schema Registry, the serializer must determine which schema to use when writing data. You can configure schema lookup options to control how the appropriate schema is resolved from the registry. The following schema lookup options are available:

• SchemaResolverProperties.FIND_LATEST_ARTIFACT—A boolean flag that determines whether your serializer should attempt to locate the most recent artifact in the registry for the given groupId and artifactId combination configured in the configuration property map. The default value is false.

  config.put(SchemaResolverProperties.FIND_LATEST_ARTIFACT, true);

• SchemaResolverProperties.EXPLICIT_ARTIFACT_VERSION—A string value that represents the artifact version used for querying an artifact in the registry. This property overrides the version returned by the ArtifactReferenceResolverStrategy. If this property and FIND_LATEST_ARTIFACT are both set, this property takes precedence. The default value is null.

  config.put(SchemaResolverProperties.EXPLICIT_ARTIFACT_VERSION, "1.0.0");

• SchemaResolverProperties.EXPLICIT_ARTIFACT_ARTIFACT_ID—A string value that represents the artifactId used for querying an artifact in the registry. This property overrides the artifactId returned by the ArtifactReferenceResolverStrategy. The default value is null.

  config.put(SchemaResolverProperties.EXPLICIT_ARTIFACT_ARTIFACT_ID, "my-schema");

• SchemaResolverProperties.EXPLICIT_ARTIFACT_GROUP_ID—A string value that represents the groupId used for querying an artifact in the registry. This property overrides the groupId returned by the ArtifactReferenceResolverStrategy. The default value is null.

  config.put(SchemaResolverProperties.EXPLICIT_ARTIFACT_GROUP_ID, "com.example");

• SchemaResolverProperties.REQUEST_ATTEMPTS—Specifies the number of attempts to make when communicating with the schema registry before giving up. Valid values are any number between 1 and Long.MAX_VALUE. When used with USE_CACHED_ON_ERROR, this property specifies the number of attempts before falling back to the last cached value. The default value is 3.

  config.put(SchemaResolverProperties.REQUEST_ATTEMPTS, 5);

• SchemaResolverProperties.REQUEST_ATTEMPT_BACKOFF_MS—Specifies the backoff time in milliseconds between retry attempts when communicating with the schema registry. Valid values are any number between 0 and Long.MAX_VALUE. You can set this value as a Long, string or a Duration object. The default value is 500 milliseconds.

  // Use a long value
  config.put(SchemaResolverProperties.REQUEST_ATTEMPT_BACKOFF_MS, 500L);
  
  // Use a string
  config.put(SchemaResolverProperties.REQUEST_ATTEMPT_BACKOFF_MS, "500");
  
  // Use a Duration object
  config.put(SchemaResolverProperties.REQUEST_ATTEMPT_BACKOFF_MS, Duration.ofMillis(500));

Auto-Registration Configuration

Auto-registration allows serializers to automatically register schemas in Solace Schema Registry when they don't exist during serialization operations. This feature simplifies schema management by eliminating the need to manually register schemas before using them in your applications. When auto-registration is enabled, the serializer will attempt to register the schema if it cannot find a matching schema in the registry for the given artifact reference.

Auto-registration is particularly useful in development environments where schemas are frequently updated, or in scenarios where you want to streamline the deployment process by allowing applications to self-register their schemas. However, in production environments, you may want to disable auto-registration to maintain strict control over schema evolution and prevent unauthorized schema modifications.

• SchemaResolverProperties.AUTO_REGISTER_ARTIFACT—A boolean flag that controls whether schemas should be automatically registered when they don't exist in the registry during serialization. When set to true, the serializer will attempt to register the schema if it's not found in the registry. When set to false, the default, the serializer will throw an exception if the schema is not found. This property only affects serialization operations.

  // Enable auto-registration of schemas. This property is false by default.
  config.put(SchemaResolverProperties.AUTO_REGISTER_ARTIFACT, true);  
  

• SchemaResolverProperties.AUTO_REGISTER_ARTIFACT_IF_EXISTS—Controls the behavior when auto-registering artifacts that might already exist in the registry. This property works in conjunction with AUTO_REGISTER_ARTIFACT and only takes effect when auto-registration is enabled. The available options are:
  • IfArtifactExists.CREATE_VERSION—If a schema with the same content already exists in the registry, create a new version of that schema. This is useful when you want to maintain version history even for identical schemas.

    config.put(SchemaResolverProperties.AUTO_REGISTER_ARTIFACT, true);
    
    // Set behavior to create a new version when artifact exists
    config.put(SchemaResolverProperties.AUTO_REGISTER_ARTIFACT_IF_EXISTS,
    SchemaResolverProperties.IfArtifactExists.CREATE_VERSION);
    

  • IfArtifactExists.FAIL—If a schema with the same name already exists in the registry, the registration will fail and throw a SerializationException. This provides strict control over schema registration.

    config.put(SchemaResolverProperties.AUTO_REGISTER_ARTIFACT, true);
    
    // Set behavior to fail when artifact exists
    config.put(SchemaResolverProperties.AUTO_REGISTER_ARTIFACT_IF_EXISTS,
    SchemaResolverProperties.IfArtifactExists.FAIL);
    

  • IfArtifactExists.FIND_OR_CREATE_VERSION—If a schema with the same content already exists in the registry, use the existing schema. Otherwise, create a new version. This is the default behavior and provides the most flexible approach to schema management.

    config.put(SchemaResolverProperties.AUTO_REGISTER_ARTIFACT, true);
    
    // Set behavior to find existing or create new version (default)
    config.put(SchemaResolverProperties.AUTO_REGISTER_ARTIFACT_IF_EXISTS,
    SchemaResolverProperties.IfArtifactExists.FIND_OR_CREATE_VERSION);
    

JSON Schema Auto-Registration

When using auto-registration with JSON Schema serialization, you must also specify the schema location using SchemaResolverProperties.SCHEMA_LOCATION to provide the classpath resource path to the schema file. Unlike Avro schemas, which embed schema information in the serialized data, JSON Schema requires an explicit schema file reference for auto-registration to work properly.

The following example shows you how to configure the schema location:

// Enable auto-registration
config.put(SchemaResolverProperties.AUTO_REGISTER_ARTIFACT, true);

// Specify the classpath resource path to the schema to be uploaded by auto-register
config.put(SchemaResolverProperties.SCHEMA_LOCATION, "json-schema/user.json");

SERDES Header Configuration

The SCHEMA_HEADER_IDENTIFIERS property allows you to select different SERDES header formats for schema identification in message headers. This configuration involves a tradeoff between efficiency and interoperability, enabling you to optimize your messaging system based on your specific requirements and integration scenarios.

When a message producer sends a message, the serializer component encodes the data according to a registered schema and includes a schema identifier in the message headers. The format of this schema identifier can be configured to balance performance optimization with cross-protocol compatibility needs.

The SCHEMA_HEADER_IDENTIFIERS property accepts values from the SchemaHeaderId enum, which provides two header format options:

• SchemaHeaderId.SCHEMA_ID(default)—Efficient long header configuration optimized for performance.
• SchemaHeaderId.SCHEMA_ID_STRING—String header configuration for optimal interoperability across protocols.

Efficiency vs. Interoperability Considerations

The choice between header formats represents a tradeoff:

• Efficiency—The default SCHEMA_ID option provides the best performance and efficiency by using a compact 8-byte long value for schema identification. This binary format minimizes header overhead and processing time, making it ideal for high-throughput Solace-to-Solace messaging scenarios.
• Interoperability—The SCHEMA_ID_STRING option ensures optimal compatibility between different messaging protocols by using human-readable string values. While this introduces some additional header processing overhead, it enables message translation and consumption across different messaging protocol environments. REST messaging serves as a specific example where string-only headers enable easier interoperability between protocols. When messages need to be consumed by REST endpoints or translated between different messaging systems, string-based schema identifiers provide better compatibility and easier debugging capabilities.

Configuration Examples

The following example shows how to configure the SCHEMA_HEADER_IDENTIFIERS property:

// Use SCHEMA_ID, the default, for maximum efficiency
config.put(SerdeProperties.SCHEMA_HEADER_IDENTIFIERS, SchemaHeaderId.SCHEMA_ID);

// Use SCHEMA_ID_STRING for optimal interoperability
config.put(SerdeProperties.SCHEMA_HEADER_IDENTIFIERS, SchemaHeaderId.SCHEMA_ID_STRING);

Avro-Specific Serializer and Deserializer Configuration

Apache Avro is a data serialization system that provides a compact binary data format and schema evolution capabilities. Avro serialization allows you to encode and decode complex data structures efficiently—encoding before publishing them as messages and decoding when consuming messages. This ensures consistent data formatting across your messaging applications and enables schema evolution as your application requirements change.

This section covers Avro-specific configuration properties, for common SERDES configuration properties, including setting the required REGISTRY_URL property, see Common Serializer and Deserializer Configuration.

The following Java imports are required for implementing Avro serialization and deserialization using the SERDES Collection:

import com.solace.serdes.avro.AvroSerializer;
import com.solace.serdes.avro.AvroDeserializer;
import org.apache.avro.Schema;
import org.apache.avro.SchemaParser;
import org.apache.avro.generic.GenericData;
import org.apache.avro.generic.GenericRecord;
import org.apache.avro.generic.GenericRecord;
import java.util.HashMap;
import java.util.Map;

You can set all SERDES properties in a HashMap object with the put() method:

Map<String, Object> config = new HashMap<>();
// set all your configuration parameters with config.put()

• Dereferencing Schemas
• Avro Encoding Types
• Record ID Artifact Resolver Strategy

Dereferencing Schemas

The DEREFERENCED_SCHEMA property controls how serializers handle Avro schemas that contain references to other schemas. In Avro schema design, there are two primary approaches to managing complex schemas:

• Referenced schemas—These schemas are structured to reference or include other shared schema components, enabling a modular and reusable design. Common types can be defined once and reused across multiple schemas. Referenced schemas are reusable and easy to maintain.
• Dereferenced schemas—These schemas, also called flat schemas, have all references expanded inline, creating a single, self-contained definitions with no external dependencies. Dereferenced schemas simplify consumption by eliminating the need to resolve multiple schema references at runtime.

When set to true (the default), the DEREFERENCED_SCHEMA property tells the serializer to treat the schema in the record as fully dereferenced and use it directly for registry lookups. The serializer does not attempt to extract or manage referenced sub-schemas, which simplifies schema handling in your application.

This configuration is useful in scenarios where you want to:

• simplify schema handling in your application by avoiding schema reference resolution.
• optimize performance by reducing the number of schema registry lookups.

When DEREFERENCED_SCHEMA is set to false, the serializer will treat the provided schema as one that needs to be decomposed into reference schemas. This allows applications to use one Avro schema file to represent multiple Avro schemas in the registry, which can reduce storage of common sub-schemas in the registry.

Here's how to configure a serializer to decompose schemas into references by setting DEREFERENCED_SCHEMA to false:

config.put(AvroProperties.DEREFERENCED_SCHEMA, false);

Avro Encoding Types

The ENCODING_TYPE property allows you to specify the encoding format used by the AvroSerializer to convert data to bytes. Avro supports two primary encoding formats, each with different characteristics and use cases:

• AvroEncoding.BINARY—Uses Avro's DirectBinaryEncoder to produce a compact binary representation of the data. This is the default encoding and provides the most efficient serialization in terms of message size and processing performance.
• AvroEncoding.JSON—Uses Avro's JsonEncoder to produce a human-readable JSON representation of the data. While less efficient than binary encoding, this format is useful for debugging, logging, and interoperability with systems that expect JSON.

The encoding type affects only the wire format of the serialized data; it does not change how the schema is resolved or how the data is structured. Both encoding types maintain full compatibility with the Avro schema system.

The encoding type is set as an Avro SERDES message header. This impacts the AvroDeserializer, which reads the assigned header value and attempts to use the appropriate decoder. If no header is present, the AvroDeserializer assumes the message was encoded with binary and attempts to decode it accordingly.

Here's how to configure a serializer to use JSON encoding:

config.put(AvroProperties.ENCODING_TYPE, AvroEncoding.JSON);

When choosing an encoding type, consider these factors:

• Use BINARY encoding when optimizing for performance, bandwidth efficiency, or message throughput.
• Use JSON encoding when debugging applications or when human readability is important.

Record ID Artifact Resolver Strategy

The RecordIdStrategy automatically determines which schema to use during serialization by extracting information directly from the record's payload. It uses the schema name as the artifactId and the schema namespace as the groupId to construct the complete ArtifactReference. This approach eliminates the need for explicit schema specification on a per message basis because it creates a direct mapping between the data structure of your records and schema artifacts stored in the registry. For successful implementation, Solace Schema Registry must contain schemas with an artifactId and a groupId that corresponds to the schema name and namespace used in your application's data models. The strategy is valuable in systems where the schema information is inherently contained within the record structure, as it reduces configuration overhead and ensures that the correct schema is always used for each record type.

In summary, the RecordIdStrategy:

• Extracts the schema from the record's payload.
• Uses the schema name as the artifactId in the ArtifactReference.
• Uses the schema namespace as the groupId in the ArtifactReference.
• Uses the constructed ArtifactReference to locate the correct schema in Solace Schema Registry.

Here's how to configure a serializer to use the RecordIdStrategy:

config.put(SchemaResolverProperties.ARTIFACT_RESOLVER_STRATEGY, RecordIdStrategy.class);

JSON Schema-Specific Serializer and Deserializer Configuration

JSON Schema is a vocabulary that defines how to annotate and validate JSON documents. JSON Schema serialization allows you to encode and decode structured JSON data efficiently—encoding before publishing them as messages and decoding when consuming messages. This ensures consistent data formatting across your messaging applications and enables schema evolution as your application requirements change.

This section covers JSON Schema-specific configuration properties. For common SERDES configuration properties, including setting the required REGISTRY_URL property, see Common Serializer and Deserializer Configuration.

The following Java imports are required for implementing JSON Schema serialization and deserialization using the SERDES Collection:

import com.solace.serdes.jsonschema.JsonSchemaSerializer;
import com.solace.serdes.jsonschema.JsonSchemaDeserializer;
import com.solace.serdes.jsonschema.JsonSchemaProperties;
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import java.util.HashMap;
import java.util.Map;

You can set all SERDES properties in a HashMap object with the put() method:

Map<String, Object> config = new HashMap<>();
// set all your configuration parameters with config.put()

• JSON Schema Validation
• Java Type Property Configuration

JSON Schema Validation

The VALIDATE_SCHEMA property controls whether JSON schema validation is performed during serialization and deserialization operations. When enabled, the serializer and deserializer will validate JSON data against its registered schema to ensure data conformity and catch schema violations early in the processing pipeline.

Schema validation provides several benefits:

• Data integrity—Ensures that JSON data conforms to the expected schema structure before processing.
• Early error detection—Catches schema violations during serialization/deserialization rather than downstream processing.
• Consistency enforcement—Guarantees that all messages follow the defined data contract.

However, enabling validation does introduce some performance overhead, so you may want to disable it in performance-critical scenarios where data integrity is guaranteed by other means.

The VALIDATE_SCHEMA property accepts a boolean value:

• true (default)—Enables JSON schema validation during serialization and deserialization operations.
• false—Disables JSON schema validation, improving performance but removing data integrity checks.

Here's how to configure JSON schema validation:

// Enable JSON schema validation (default behavior)
config.put(JsonSchemaProperties.VALIDATE_SCHEMA, true);

// Disable JSON schema validation for improved performance
config.put(JsonSchemaProperties.VALIDATE_SCHEMA, false);

Java Type Property Configuration

The TYPE_PROPERTY configuration specifies the JSON schema property name that contains the Java type class path for deserialization. The JsonSchemaDeserializer uses this property to determine the target Java class when converting JSON data back to Java objects, enabling proper type reconstruction during the deserialization process.

This property is particularly important when:

• Your JSON schema includes type information for Java object reconstruction.
• You need to deserialize JSON data into specific Java classes rather than generic JSON objects.
• Your application uses polymorphic types that require runtime type resolution.

The TYPE_PROPERTY accepts a string value that specifies the property name in your JSON schema:

• Default value: "javaType"—The deserializer will look for a property named "javaType" in the JSON schema.
• Custom value: Any valid JSON property name that contains the Java class path information.

Here's how to configure the Java type property:

// Use the default "javaType" property name
config.put(JsonSchemaProperties.TYPE_PROPERTY, "javaType");

// Use a custom property name for Java type information
config.put(JsonSchemaProperties.TYPE_PROPERTY, "className");

// Use another custom property name
config.put(JsonSchemaProperties.TYPE_PROPERTY, "targetClass");

The following example shows a JSON schema that includes the javaType property. When the JsonSchemaDeserializer processes JSON data conforming to this schema, it will use the javaType value (com.example.User) to instantiate the correct Java class during deserialization:

{
  "$schema": "https://json-schema.org/draft-07/schema",
  "type": "object",
  "javaType": "com.example.User",
  "properties": {
    "name": {
      "type": "string"
    },
    "email": {
      "type": "string",
      "format": "email"
    },
    "age": {
      "type": "integer",
      "minimum": 0
    }
  },
  "required": ["name", "email"]
}

Serializing and Deserializing Messages with Avro

The example below shows you how to create a configuration map, an AvroSerializer object, and an AvroDeserializer object:

// 1. Set SERDES properties using a configuration HashMap
Map<String, Object> config = new HashMap<>();  // use config.put(property, value) to set SchemaResolverProperties, AvroProperties 

// 2. Create and configure your serializer object
Serializer<GenericRecord> serializer = new AvroSerializer<>();
serializer.configure(config);

// 3. Create and configure your deserializer object
Deserializer<GenericRecord> deserializer = new AvroDeserializer<>();
deserializer.configure(config);

The following sections explain how to serialize and deserialize messages with the PubSub+ JCSMP API:

• Serializing and Sending Avro Messages using the PubSub+ JCSMP API
• Receiving and Deserializing Avro Messages using the PubSub+ JCSMP API

Serializing and Sending Avro Messages using the PubSub+ JCSMP API

When you are serializing and sending messages using the PubSub+ JCSMP API, first create a GenericRecord that conforms to your Avro schema, then serialize and send the message. The example below shows you how to create a simple user record, convert it to binary format with the SerdeMessage.serialize() method, and then publish the message:

GenericRecord user = initEmptyUserRecord();
user.put("name", "John Doe");
user.put("id", "123");
user.put("email", "support@solace.com");

// Serialize and send the message
BytesMessage msg = JCSMPFactory.onlyInstance().createMessage(BytesMessage.class);
SerdeMessage.serialize(serializer, topic, msg, user);
producer.send(msg, topic);

The SerdeMessage.serialize() method not only converts the data to binary format but also sets important schema information as Solace Message Format (SMF) user properties in the message. This includes the schema identifier and other metadata needed for proper deserialization. The schema information is stored in the SMF user properties, not in the payload itself, which is important for interoperability with other messaging protocols.

For a complete example, see AvroSerializeProducer.java on the Solace Samples Repository.

Receiving and Deserializing Avro Messages using the PubSub+ JCSMP API

When you are receiving and deserializing Avro serialized messages with the PubSub+ JCSMP API, you configure a message consumer to receive and deserialize messages synchronously or asynchronously:

Receive and Deserialize a Message Asynchronously with a Callback

The example below shows you how to create an XMLMessageConsumer, provide it with a deserializer as the first parameter, and configure three callback handlers that manage successful deserialization, deserialization errors, and general JCSMP exceptions:

// Create a latch to synchronize the main thread with the message consumer
CountDownLatch latch = new CountDownLatch(1);

// Set up the message consumer with a deserialization callback
XMLMessageConsumer cons = session.getMessageConsumer(Consumed.with(deserializer, (msg, genericRecord) -> {
    System.out.printf("Got record: %s%n", genericRecord);
    latch.countDown(); // Signal the main thread that a message has been received
}, (msg, deserializationException) -> {
    System.out.printf("Got exception: %s%n", deserializationException);
    System.out.printf("But still have access to the message: %s%n", msg.dump());
    latch.countDown();
}, jcsmpException -> {
    System.out.printf("Got exception: %s%n", jcsmpException);
    latch.countDown();
}));
cons.start();

// Wait for the consumer to receive the message
latch.await();

For a complete example, see HelloWorldJCSMPAvroSerde.java on the Solace Samples Repository.

Receive and Deserialize a Message Synchronously

The example below shows you how to create an XMLMessageConsumer that polls for messages in a loop, manually deserializes each message, and continues until the user terminates the program:

// Create a message consumer and subscribe to the topic
final XMLMessageConsumer consumer = session.getMessageConsumer((XMLMessageListener) null);
session.addSubscription(topic);

WaitForEnterThread exitListener = new WaitForEnterThread();
exitListener.start();

// Start the consumer and wait for a message
consumer.start();
while (!exitListener.isDone()) {
    // Try to receive a message with a 1-second timeout
    BytesXMLMessage msg = consumer.receive(1000);
    if (msg == null) continue;

    // Deserialize the received message
    GenericRecord genericRecord = SerdeMessage.deserialize(deserializer, msg);
    System.out.printf("Got message: %s%n", genericRecord);
}

exitListener.join();
session.closeSession();

For a complete example, see AvroDeserializeConsumer.java on the Solace Samples Repository.

Serializing and Deserializing Messages with JSON Schema

The example below shows you how to create a configuration map, a JsonSchemaSerializer object, and a JsonSchemaDeserializer object:

// 1. Set SERDES properties using a configuration HashMap
Map<String, Object> config = new HashMap<>();  // use config.put(property, value) to set SchemaResolverProperties, JsonSchemaProperties

// 2. Create and configure your serializer object
Serializer<JsonNode> serializer = new JsonSchemaSerializer<>();
serializer.configure(config);

// 3. Create and configure your deserializer object
Deserializer<JsonNode> deserializer = new JsonSchemaDeserializer<>();
deserializer.configure(config);

The following sections explain how to serialize and deserialize messages with the PubSub+ JCSMP API:

• Serializing and Sending JSON Schema Messages using the PubSub+ JCSMP API
• Receiving and Deserializing JSON Schema Messages using the PubSub+ JCSMP API

Serializing and Sending JSON Schema Messages using the PubSub+ JCSMP API

When you are serializing and sending messages using the PubSub+ JCSMP API, first create a JsonNode that conforms to your JSON schema, then serialize and send the message. The example below shows you how to create a simple user object, convert it to binary format with the SerdeMessage.serialize() method, and then publish the message:

ObjectMapper mapper = new ObjectMapper();
JsonNode user = mapper.createObjectNode()
    .put("name", "John Doe")
    .put("id", "123")
    .put("email", "support@solace.com");

// Serialize and send the message
BytesMessage msg = JCSMPFactory.onlyInstance().createMessage(BytesMessage.class);
SerdeMessage.serialize(serializer, topic, msg, user);
producer.send(msg, topic);

The SerdeMessage.serialize() method not only converts the data to binary format but also sets important schema information as Solace Message Format (SMF) user properties in the message. This includes the schema identifier and other metadata needed for proper deserialization. The schema information is stored in the SMF user properties, not in the payload itself, which is important for interoperability with other messaging protocols.

For a complete example, see JsonSchemaSerializeProducer.java on the Solace Samples Repository.

Receiving and Deserializing JSON Schema Messages using the PubSub+ JCSMP API

When you are receiving and deserializing JSON Schema serialized messages with the PubSub+ JCSMP API, you configure a message consumer to receive and deserialize messages synchronously or asynchronously:

Receive and Deserialize a Message Asynchronously with a Callback

The example below shows you how to create an XMLMessageConsumer, provide it with a deserializer as the first parameter, and configure three callback handlers that manage successful deserialization, deserialization errors, and general JCSMP exceptions:

// Create a latch to synchronize the main thread with the message consumer
CountDownLatch latch = new CountDownLatch(1);

// Set up the message consumer with a deserialization callback
XMLMessageConsumer cons = session.getMessageConsumer(Consumed.with(deserializer, (msg, jsonNode) -> {
    System.out.printf("Got object: %s%n", jsonNode);
    latch.countDown(); // Signal the main thread that a message has been received
}, (msg, deserializationException) -> {
    System.out.printf("Got exception: %s%n", deserializationException);
    System.out.printf("But still have access to the message: %s%n", msg.dump());
    latch.countDown();
}, jcsmpException -> {
    System.out.printf("Got exception: %s%n", jcsmpException);
    latch.countDown();
}));
cons.start();

// Wait for the consumer to receive the message
latch.await();

For a complete example, see HelloWorldJCSMPJsonSchemaSerde.java on the Solace Samples Repository.

Receive and Deserialize a Message Synchronously

The example below shows you how to create an XMLMessageConsumer that polls for messages in a loop, manually deserializes each message, and continues until the user terminates the program:

// Create a message consumer and subscribe to the topic
final XMLMessageConsumer consumer = session.getMessageConsumer((XMLMessageListener) null);
session.addSubscription(topic);

WaitForEnterThread exitListener = new WaitForEnterThread();
exitListener.start();

// Start the consumer and wait for a message
consumer.start();
while (!exitListener.isDone()) {
    // Try to receive a message with a 1-second timeout
    BytesXMLMessage msg = consumer.receive(1000);
    if (msg == null) continue;

    // Deserialize the received message
    JsonNode jsonNode = SerdeMessage.deserialize(deserializer, msg);
    System.out.printf("Got message: %s%n", jsonNode);
}

exitListener.join();
session.closeSession();

For a complete example, see JsonSchemaDeserializeConsumerToJsonNode.java on the Solace Samples Repository.