Serializing and Deserializing Messages with the Solace JCSMP API

You can use the Solace JCSMP API with the SERDES Collection to handle structured message payloads efficiently. SERDES, which stands for Serialization and Deserialization, allows applications to convert complex data structures into a compact, transmittable format when publishing messages and to reconstruct them when consuming messages. This is particularly useful when integrating with schema registries. The SERDES Collection refers to the set of serializer and deserializer libraries, such as Solace Avro SERDES for Java, Solace JSON Schema SERDES for Java, and Generic SERDES for Java. You can use these libraries to integrate applications with Solace Schema Registry for schema-based message serialization and deserialization.

Prerequisites
Deploying with Maven
Serializing and Deserializing Messages with Avro
Serializing and Deserializing Messages with JSON Schema

Prerequisites

Before implementing SERDES with the Solace JCSMP API, you should understand the language-agnostic concepts and configuration options covered in Serialization and Deserialization with Solace Schema Registry. Key topics include:

Connecting to Schema Registry (authentication and security)
Choosing schema resolution strategies (Destination ID, Topic ID, Topic ID with Profiles, Record ID)
Optimizing performance (caching and lookup options)
Advanced configuration (auto-registration, header formats, Avro and JSON Schema-specific settings)
Cross-protocol message translation

This page focuses on Solace JCSMP API-specific implementation details, including Java imports and code examples for serializing and deserializing messages.

Deploying with Maven

To use the SERDES Collection with the Solace JCSMP API, you need to include the appropriate Maven dependencies in your project. The SERDES libraries are distributed as separate artifacts that you can include based on your serialization format requirements.

The following sections show you how to configure your Maven project to include the necessary dependencies for schema registry integration:

SERDES BOM Dependencies (Recommended)
Avro Serializer Dependencies
JSON Schema Serializer Dependencies
Generic Serializer Dependencies

SERDES BOM Dependencies (Recommended)

The SERDES Bill of Materials (BOM) provides a centralized way to manage compatible versions of all Solace Schema Registry SERDES components. Using the BOM ensures that all SERDES dependencies are aligned with tested and compatible versions. This simplifies dependency management and reduces version conflicts in your applications.

The BOM includes version management for:

Avro SERDES components
JSON Schema SERDES components
Common SERDES libraries
Compatible versions of underlying dependencies

To use the SERDES BOM, add it to your Maven pom.xml file in the dependencyManagement section:

<dependencyManagement>
    <dependencies>
        <dependency>
            <groupId>com.solace</groupId>
            <artifactId>solace-schema-registry-serdes-bom</artifactId>
            <version>1.0.0</version>
            <type>pom</type>
            <scope>import</scope>
        </dependency>
    </dependencies>
</dependencyManagement>

After importing the BOM, you can add SERDES dependencies without specifying versions:

<dependencies>
    <!-- JCSMP API -->
    <dependency>
        <groupId>com.solacesystems</groupId>
        <artifactId>sol-jcsmp</artifactId>
        <version>10.29.0</version>
    </dependency>
    
    <!-- Avro SERDES (version managed by BOM) -->
    <dependency>
        <groupId>com.solace</groupId>
        <artifactId>solace-schema-registry-avro-serde</artifactId>
    </dependency>
    
    <!-- JSON Schema SERDES (version managed by BOM) -->
    <dependency>
        <groupId>com.solace</groupId>
        <artifactId>solace-schema-registry-jsonschema-serde</artifactId>
    </dependency>
</dependencies>

Avro Serializer Dependencies

The Avro serializer provides a specific implementation for Apache Avro serialization and deserialization. This dependency includes the AvroSerializer, AvroDeserializer, and Avro-specific configuration properties, and also brings in the required common SERDES components used across all schema formats.

Add the following dependency to your Maven pom.xml file:

<dependencies>
    <dependency>
        <groupId>com.solace</groupId>
        <artifactId>solace-schema-registry-avro-serde</artifactId>
        <version>1.0.0</version>
    </dependency>
</dependencies>

For the latest version information and additional details about the Avro SERDES artifact, see the Maven Central Repository.

Important considerations when adding Maven dependencies:

Always use the latest compatible versions of both the Solace JCSMP API and SERDES libraries for optimal performance and security.
The Avro SERDES dependency automatically includes the necessary Apache Avro libraries as transitive dependencies.

JSON Schema Serializer Dependencies

The JSON Schema serializer provides a specific implementation for JSON Schema serialization and deserialization. This dependency includes the JsonSchemaSerializer, JsonSchemaDeserializer, and JSON Schema-specific configuration properties, and also brings in the required common SERDES components used across all schema formats.

Add the following dependency to your Maven pom.xml file:

<dependencies>
    <dependency>
        <groupId>com.solace</groupId>
        <artifactId>solace-schema-registry-jsonschema-serde</artifactId>
        <version>1.0.0</version>
    </dependency>
</dependencies>

For the latest version information and additional details about the JSON Schema SERDES artifact, see the Maven Central Repository.

Important considerations when adding Maven dependencies:

Always use the latest compatible versions of both the Solace JCSMP API and SERDES libraries for optimal performance and security.
The JSON Schema SERDES dependency automatically includes the necessary JSON Schema validation libraries as transitive dependencies.

Generic Serializer Dependencies

The Generic SERDES for Java provides simple string serialization and deserialization capabilities with no Solace Schema Registry interaction. Generic SERDES for Java is automatically included when you update to version 10.28 or later of the Solace JCSMP API.

Add the following dependencies to your Maven pom.xml file:

<dependencies>
  <dependency>
    <groupId>com.solacesystems</groupId>
    <artifactId>sol-jcsmp</artifactId>
    <version>10.29.0</version>  
  </dependency>
</dependencies>

Serializing and Deserializing Messages with Avro

The example below shows you how to create a configuration map, an AvroSerializer object, and an AvroDeserializer object:

// 1. Set SERDES properties using a configuration HashMap
Map<String, Object> config = new HashMap<>();

// Configure schema registry connection (required)
config.put(SchemaResolverProperties.REGISTRY_URL, "http://localhost:8081/apis/registry/v3");
config.put(SchemaResolverProperties.AUTH_USERNAME, "myUsername");
config.put(SchemaResolverProperties.AUTH_PASSWORD, "myPassword");

// Configure Avro-specific properties (optional)
config.put(AvroProperties.ENCODING_TYPE, AvroProperties.AvroEncoding.BINARY);

// 2. Create and configure your serializer object
Serializer<GenericRecord> serializer = new AvroSerializer<>();
serializer.configure(config);

// 3. Create and configure your deserializer object
Deserializer<GenericRecord> deserializer = new AvroDeserializer<>();
deserializer.configure(config);

For detailed information about all available configuration properties, see Getting Started with SERDES and Avro-Specific Configuration.

Advanced Configuration Examples

The following examples demonstrate advanced SERDES configuration scenarios, including topic-to-schema mapping strategies using profiles. For conceptual information about these strategies, see Solace Topic ID Strategy with Profiles.

Topic ID Strategy with Profiles - Topic Expression Only

This example shows how to create topic mappings where the topic expression itself becomes the artifact ID. For more information about when to use this mapping type, see Solace Topic ID Strategy with Profiles.

First, configure the serializer to use the Solace Topic ID Strategy:

config.put(SchemaResolverProperties.ARTIFACT_RESOLVER_STRATEGY, SolaceTopicIdStrategy.class);

Then create and configure the topic profile with your mappings:

// Create a new instance of SolaceTopicProfile
SolaceTopicProfile profile = new SolaceTopicProfile();

// Create mappings with topic expressions
SolaceTopicArtifactMapping mapping1 = SolaceTopicArtifactMapping.create("solace/samples");
SolaceTopicArtifactMapping mapping2 = SolaceTopicArtifactMapping.create("solace/*/sample");
SolaceTopicArtifactMapping mapping3 = SolaceTopicArtifactMapping.create("solace/>");

// Add mappings to profile
profile.add(mapping1);
profile.add(mapping2);
profile.add(mapping3);

// Add the profile to your serializer configuration
config.put(SchemaResolverProperties.STRATEGY_TOPIC_PROFILE, profile);

Topic ID Strategy with Profiles - Custom Artifact ID

This example shows how to map topic expressions to specific custom artifact IDs. For more information about when to use this mapping type, see Solace Topic ID Strategy with Profiles.

First, configure the serializer to use the Solace Topic ID Strategy:

config.put(SchemaResolverProperties.ARTIFACT_RESOLVER_STRATEGY, SolaceTopicIdStrategy.class);

Then create and configure the topic profile with your custom artifact ID mappings:

// Create a new instance of SolaceTopicProfile
SolaceTopicProfile profile = new SolaceTopicProfile();

// Create mappings with topic expressions and custom artifact IDs
SolaceTopicArtifactMapping mapping1 = SolaceTopicArtifactMapping.create("solace/samples", "User");
SolaceTopicArtifactMapping mapping2 = SolaceTopicArtifactMapping.create("solace/*/sample", "NewUser");
SolaceTopicArtifactMapping mapping3 = SolaceTopicArtifactMapping.create("solace/>", "OldUser");

// Add mappings to profile
profile.add(mapping1);
profile.add(mapping2);
profile.add(mapping3);

// Add the profile to your serializer configuration
config.put(SchemaResolverProperties.STRATEGY_TOPIC_PROFILE, profile);

Topic ID Strategy with Profiles - Full ArtifactReference

This example shows how to create complete artifact references with groupId, artifactId, and optional version. For more information about when to use this mapping type, see Solace Topic ID Strategy with Profiles.

First, configure the serializer to use the Solace Topic ID Strategy:

config.put(SchemaResolverProperties.ARTIFACT_RESOLVER_STRATEGY, SolaceTopicIdStrategy.class);

Then create and configure the topic profile with full artifact references:

// Create a new instance of SolaceTopicProfile
SolaceTopicProfile profile = new SolaceTopicProfile();

// Create artifact reference
ArtifactReferenceBuilder builder = new ArtifactReferenceBuilder();
ArtifactReference reference = builder
        .groupId("com.solace.samples.serdes.avro.schema")
        .artifactId("User")
        .build();

// Create mapping with topic expression and artifact reference
SolaceTopicArtifactMapping mapping = SolaceTopicArtifactMapping.create("solace/samples", reference);

// Create mapping with version-specific reference
ArtifactReference versionedReference = builder
        .groupId("com.solace.samples.serdes.avro.schema")
        .artifactId("User")
        .version("0.0.1")
        .build();

SolaceTopicArtifactMapping versionedMapping = SolaceTopicArtifactMapping.create("solace/>", versionedReference);

// Add mappings to profile
profile.add(mapping);
profile.add(versionedMapping);

// Add the profile to your serializer configuration
config.put(SchemaResolverProperties.STRATEGY_TOPIC_PROFILE, profile);

The following sections explain how to serialize and deserialize messages with the Solace JCSMP API:

Serializing and Sending Avro Messages using the Solace JCSMP API
Receiving and Deserializing Avro Messages using the Solace JCSMP API

Serializing and Sending Avro Messages using the Solace JCSMP API

When you are serializing and sending messages using the Solace JCSMP API, first create a GenericRecord that conforms to your Avro schema, then serialize and send the message. The example below shows you how to create a simple user record, convert it to binary format with the SerdeMessage.serialize() method, and then publish the message:

GenericRecord user = initEmptyUserRecord();
user.put("name", "John Doe");
user.put("id", "123");
user.put("email", "support@solace.com");

// Serialize and send the message
BytesMessage msg = JCSMPFactory.onlyInstance().createMessage(BytesMessage.class);
SerdeMessage.serialize(serializer, topic, msg, user);
producer.send(msg, topic);

The SerdeMessage.serialize() method not only converts the data to binary format but also sets important schema information as Solace Message Format (SMF) user properties in the message. This includes the schema identifier and other metadata needed for proper deserialization. The schema information is stored in the SMF user properties, not in the payload itself, which is important for interoperability with other messaging protocols.

For a complete example, see AvroSerializeProducer.java on the Solace Samples Repository.

For a complete list of Avro SERDES properties and methods, see the Java Avro SERDES API Reference.

Receiving and Deserializing Avro Messages using the Solace JCSMP API

When you are receiving and deserializing Avro serialized messages with the Solace JCSMP API, you configure a message consumer to receive and deserialize messages synchronously or asynchronously:

Receive and Deserialize a Message Asynchronously with a Callback

The example below shows you how to create an XMLMessageConsumer, provide it with a deserializer as the first parameter, and configure three callback handlers that manage successful deserialization, deserialization errors, and general JCSMP exceptions:

// Create a latch to synchronize the main thread with the message consumer
CountDownLatch latch = new CountDownLatch(1);

// Set up the message consumer with a deserialization callback
XMLMessageConsumer cons = session.getMessageConsumer(Consumed.with(deserializer, (msg, genericRecord) -> {
    System.out.printf("Got record: %s%n", genericRecord);
    latch.countDown(); // Signal the main thread that a message has been received
}, (msg, deserializationException) -> {
    System.out.printf("Got exception: %s%n", deserializationException);
    System.out.printf("But still have access to the message: %s%n", msg.dump());
    latch.countDown();
}, jcsmpException -> {
    System.out.printf("Got exception: %s%n", jcsmpException);
    latch.countDown();
}));
cons.start();

// Wait for the consumer to receive the message
latch.await();

For a complete example, see HelloWorldJCSMPAvroSerde.java on the Solace Samples Repository.

Receive and Deserialize a Message Synchronously

The example below shows you how to create an XMLMessageConsumer that polls for messages in a loop, manually deserializes each message, and continues until the user terminates the program:

// Create a message consumer and subscribe to the topic
final XMLMessageConsumer consumer = session.getMessageConsumer((XMLMessageListener) null);
session.addSubscription(topic);

WaitForEnterThread exitListener = new WaitForEnterThread();
exitListener.start();

// Start the consumer and wait for a message
consumer.start();
while (!exitListener.isDone()) {
    // Try to receive a message with a 1-second timeout
    BytesXMLMessage msg = consumer.receive(1000);
    if (msg == null) continue;

    // Deserialize the received message
    GenericRecord genericRecord = SerdeMessage.deserialize(deserializer, msg);
    System.out.printf("Got message: %s%n", genericRecord);
}

exitListener.join();
session.closeSession();

For a complete example, see AvroDeserializeConsumer.java on the Solace Samples Repository.

For a complete list of Avro SERDES properties and methods, see the Java Avro SERDES API Reference.

Serializing and Deserializing Messages with JSON Schema

The example below shows you how to create a configuration map, a JsonSchemaSerializer object, and a JsonSchemaDeserializer object:

// 1. Set SERDES properties using a configuration HashMap
Map<String, Object> config = new HashMap<>();

// Configure schema registry connection (required)
config.put(SchemaResolverProperties.REGISTRY_URL, "http://localhost:8081/apis/registry/v3");
config.put(SchemaResolverProperties.AUTH_USERNAME, "myUsername");
config.put(SchemaResolverProperties.AUTH_PASSWORD, "myPassword");

// Configure JSON Schema-specific properties (optional)
config.put(JsonSchemaProperties.VALIDATE_SCHEMA, true);

// 2. Create and configure your serializer object
Serializer<JsonNode> serializer = new JsonSchemaSerializer<>();
serializer.configure(config);

// 3. Create and configure your deserializer object
Deserializer<JsonNode> deserializer = new JsonSchemaDeserializer<>();
deserializer.configure(config);

For detailed information about all available configuration properties, see Getting Started with SERDES and JSON Schema-Specific Configuration.

The following sections explain how to serialize and deserialize messages with the Solace JCSMP API:

Serializing and Sending JSON Schema Messages using the Solace JCSMP API
Receiving and Deserializing JSON Schema Messages using the Solace JCSMP API

Serializing and Sending JSON Schema Messages using the Solace JCSMP API

When you are serializing and sending messages using the Solace JCSMP API, first create a JsonNode that conforms to your JSON schema, then serialize and send the message. The example below shows you how to create a simple user object, convert it to binary format with the SerdeMessage.serialize() method, and then publish the message:

ObjectMapper mapper = new ObjectMapper();
JsonNode user = mapper.createObjectNode()
    .put("name", "John Doe")
    .put("id", "123")
    .put("email", "support@solace.com");

// Serialize and send the message
BytesMessage msg = JCSMPFactory.onlyInstance().createMessage(BytesMessage.class);
SerdeMessage.serialize(serializer, topic, msg, user);
producer.send(msg, topic);

For a complete example, see JsonSchemaSerializeProducer.java on the Solace Samples Repository.

For a complete list of JSON Schema SERDES properties and methods, see the Java JSON SERDES API Reference.

Receiving and Deserializing JSON Schema Messages using the Solace JCSMP API

When you are receiving and deserializing JSON Schema serialized messages with the Solace JCSMP API, you configure a message consumer to receive and deserialize messages synchronously or asynchronously:

Receive and Deserialize a Message Asynchronously with a Callback

// Create a latch to synchronize the main thread with the message consumer
CountDownLatch latch = new CountDownLatch(1);

// Set up the message consumer with a deserialization callback
XMLMessageConsumer cons = session.getMessageConsumer(Consumed.with(deserializer, (msg, jsonNode) -> {
    System.out.printf("Got object: %s%n", jsonNode);
    latch.countDown(); // Signal the main thread that a message has been received
}, (msg, deserializationException) -> {
    System.out.printf("Got exception: %s%n", deserializationException);
    System.out.printf("But still have access to the message: %s%n", msg.dump());
    latch.countDown();
}, jcsmpException -> {
    System.out.printf("Got exception: %s%n", jcsmpException);
    latch.countDown();
}));
cons.start();

// Wait for the consumer to receive the message
latch.await();

For a complete example, see HelloWorldJCSMPJsonSchemaSerde.java on the Solace Samples Repository.

Receive and Deserialize a Message Synchronously

The example below shows you how to create an XMLMessageConsumer that polls for messages in a loop, manually deserializes each message, and continues until the user terminates the program:

// Create a message consumer and subscribe to the topic
final XMLMessageConsumer consumer = session.getMessageConsumer((XMLMessageListener) null);
session.addSubscription(topic);

WaitForEnterThread exitListener = new WaitForEnterThread();
exitListener.start();

// Start the consumer and wait for a message
consumer.start();
while (!exitListener.isDone()) {
    // Try to receive a message with a 1-second timeout
    BytesXMLMessage msg = consumer.receive(1000);
    if (msg == null) continue;

    // Deserialize the received message
    JsonNode jsonNode = SerdeMessage.deserialize(deserializer, msg);
    System.out.printf("Got message: %s%n", jsonNode);
}

exitListener.join();
session.closeSession();

For a complete example, see JsonSchemaDeserializeConsumerToJsonNode.java on the Solace Samples Repository.

For a complete list of JSON Schema SERDES properties and methods, see the Java JSON SERDES API Reference.

Provide feedback