Context Propagation for Distributed Tracing

Distributed tracing allows your enterprise applications to trace the flow of messages as they travel from your publisher, through the event mesh and to the receiving application. For a detailed overview see Distributed Tracing . For information about version requirements, see Distributed Tracing Version Compatibility.

  • The PubSub+ OpenTelemetry API Libraries support W3C propagators only.
  • For information about configuring OpenTelemetry SDK environment variables see OpenTelemetry SDK Configuration.
  • By default, traces include command line parameters visible to backend applications like Jaeger. It is important to disable this feature for security purposes because these parameters may contain sensitive information such as your user name and password. For instructions, see Disabling Automatic Resource Providers in the OpenTelemetry documentation in GitHub.

Instrumenting JMS for Distributed Tracing

There are two ways to integrate distributed tracing into an application using the PubSub+ JMS API:

  • Automatic Instrumentation allows your applications to inject and extract standard OpenTelemetry data without any modifications to your source code.
  • Manual Instrumentation involves making changes to your enterprise application's source code. These code modifications allow you to inject and extract additional context, such as Baggage and Trace States, into messages.

Automatic Instrumentation

Automatic instrumentation does not require any application code changes to send telemetry data. You do not need to change the start up parameters in your application or implement any additional libraries. To use automatic instrumentation:

  1. Download the .jar files from the following locations, and place them in your project:
  2. At the command prompt, enter the following command to configure your publishing application to inject OpenTelemetry data into published messages. To allow your application to send trace data to an OTLP endpoint you’ll need to update the property otel.exporter.otlp.endpoint=http://localhost:4317 to point to your collector's IP address:
    3. java -javaagent:<absolute-path-to-otel-jar>/opentelemetry-javaagent-all-<otel-java-version>.jar \
-Dotel.javaagent.extensions=<absolute-path-to-solace-jar>/solace-opentelemetry-jms-integration-<solace-opentelementry-version>.jar \
-Dotel.propagators=solace_jms_tracecontext \
-Dotel.exporter.otlp.endpoint=<http://localhost:4317> \
-Dotel.traces.exporter=otlp \
-Dotel.metrics.exporter=none \
-Dotel.instrumentation.jms.enabled=true \
-Dotel.resource.attributes="service.name=SolaceJMSPublisher" \
-jar <your-jms-publisher-application>.jar

    Where :

    • <absolute-path-to-otel-jar>is the absolute file path to the directory containing your OpenTelemetry Java Agent JAR file.
    • <absolute-path-to-solace-jar> is the absolute file path to the directory containing your Solace PubSub+ OpenTelemetry JMS Integration JAR file.
    • <http://localhost:4317> is the URL of the Collector you are using. The example localhost:4317is a default port for an OpenTelemetry Collector that is running locally.
    • <your-jms-publisher-application> is the name of the JAR file associated with your JMS publisher application.
    • <otel-java-version> is the version OpenTelementry Instrumentation Java Library that is compatible with the PubSub+ OpenTelemetry Integration for Solace JMS API version(<solace-opentelementry-version> ). For information about the compatibility, see Distributed Tracing Version Compatibility.
    • <solace-opentelementry-version> is the PubSub+ OpenTelemetry Integration for Solace JMS to use with the OpenTelementry Instrumentation Library (<otel-java-version>). For information about the compatibility, see Distributed Tracing Version Compatibility.
  3. At the command prompt, enter following command to configure your queue receiver application to extract OpenTelemetry data from received messages: 

    4. java -javaagent:<absolute-path-to-otel-jar>/opentelemetry-javaagent-all-<otel-java-version>.jar \
-Dotel.javaagent.extensions=<absolute-path-to-solace-jar>/solace-opentelemetry-jms-integration-<solace-opentelementry-version>.jar \
-Dotel.propagators=solace_jms_tracecontext \
-Dotel.traces.exporter=otlp \
-Dotel.metrics.exporter=none \
-Dotel.instrumentation.jms.enabled=true \
-Dotel.resource.attributes="service.name=SolaceJMSQueueSubscriber" \
-jar <your-jms-queue-receiver-application>.jar

    Where :

    • <absolute-path-to-otel-jar> is the absolute file path to the directory containing your OpenTelemetry Java Agent JAR file.
    • <absolute-path-to-solace-jar> is the absolute file path to the directory containing your PubSub+ OpenTelemetry JMS Integration JAR file.
    • <your-jms-queue-receiver-application> is the name of the JAR file associated with your JMS queue receiver application.
    • <otel-java-version> is the version OpenTelementry Instrumentation Java Library (such as 1.29.0) that is compatible with the PubSub+ OpenTelemetry Integration for Solace JMS API version(<solace-opentelementry-version> ). For information about the compatibility, see Distributed Tracing Version Compatibility.
    • <solace-opentelementry-version> is the PubSub+ OpenTelemetry Integration for Solace JMS (such as 1.1.0) to use with the OpenTelementry Instrumentation Library (<otel-java-version>). For information about the compatibility, see Distributed Tracing Version Compatibility.

When you use automatic instrumentation, the latest version of the opentelemetry-javaagent-all-<otel-java-version>.jar and solace-opentelemetry-jms-integration-<solace-opentelementry-version>.jar should not be added as Maven or Gradle dependencies in your applications. For information about the compatibility, see Distributed Tracing Version Compatibility.

For auto-instrumentation, Solace recommends that you use the OpenTelemetry Instrumentation for Java library, version 1.29, which Solace has tested to work with PubSub+ products that support auto-instrumentation. Earlier versions of the library may not be compatible with PubSub+ products. For version details, see https://github.com/open-telemetry/opentelemetry-java-instrumentation/releases.

For a more detailed example about how to set up automatic instrumentation in the JMS API see Getting Started with Solace Distributed Tracing and Context Propagation.

Manual Instrumentation

Manual Instrumentation involves making changes to your enterprise application's source code, and allows you to inject and extract additional context, such as baggage and trace states, into messages. Context propagation makes it easy to debug and optimize your application. For more information about context propagation in Solace event messages, see Distributed Tracing Context Propagation. The following examples show you how to create spans using the OpenTelemetry API.

Understanding How Context Propagation Enables Distributed Tracing in the JMS PubSub+ API

In your client application, you can use the OpenTelemetry API to create a span, which contains metadata about an operation in a distributed system. This span is associated with a context, which includes a unique TraceID. Next, when you use a PubSub+ message producer to publish a message, the Solace OTel integration package injects the context, which contains the TraceID, into the message. As the message travels through the event broker and is received by a consuming application, spans are generated at each step and have the same TraceID present in the original message context. When each span is closed in the publishing or consuming application, the Java OpenTelemetry API sends it to an OpenTelemetry collector, which collects, processes and exports the spans to a backend application that correlates the spans using their unique TraceID. A backend application uses the correlated spans to create a trace, which is an end-to-end snapshot detailing how the message traveled through the distributed system. If you do not use context propagation, then backend applications cannot use a unique TraceID to link the spans, making it difficult to trace the flow of messages through the distributed system.

Dependencies

To enable context propagation for distributed tracing, you must first add the Solace PubSub+ OpenTelemetry Integration For Solace JMS API as a dependency in your application. This library automatically adds the OpenTelemetry API and SDK libraries, which are required for context propagation. For OpenTelemetry version compatibility see Distributed Tracing Version Compatibility. Adding the libraries gives you access to the following two interfaces:

  • SolaceJmsW3CTextMapSetter— This interface allows a TextMapPropagator to inject context into a message on the publisher side.
  • SolaceJmsW3CTextMapGetter— This interface allows a TextMapPropagator to extract context from a message on the receiver side.

This guide assumes you are familiar with configuring an instance of the OpenTelemetry class. For instructions for configuring OpenTelemetry objects, see OpenTelemetry Manual Instrumentation in Java in the OpenTelemetry documentation.

Generating a Send Span on Message Publish

Your publishing application can generate a send span and export it to the OpenTelemetry Collector. The following steps show you how to inject context into a message and generate a send span for a published message:

  1. Create a new span and set span attributes with the setAttribute() method. Next, set the current context as the parent of this span. Start the span with the startSpan() method:
    2. final Span sendSpan = tracer
	.spanBuilder("mySolacePublisherApp" + " " + MessagingOperationValues.PROCESS)
	.setSpanKind(SpanKind.CLIENT)
	//Publish to a non-temporary topic endpoint
	.setAttribute(SemanticAttributes.MESSAGING_DESTINATION_KIND, MessagingDestinationKindValues.TOPIC)
	.setAttribute(SemanticAttributes.MESSAGING_TEMP_DESTINATION, false)
	//Set more attributes as needed
	//.setAttribute(...)
	//.setAttribute(...)
	.setParent(Context.current()) //Set the current context as the parent span
	.startSpan();

  2. Set the created span from Step 1 (sendSpan in this example) as the new current context. Next, inject the current context into your message and then publish the message. Call the end() method on the span to export the span data:

    3. try (Scope scope = sendSpan.makeCurrent()) {
	final SolaceJmsW3CTextMapSetter setter = new SolaceJmsW3CTextMapSetter();
	final TextMapPropagator propagator = openTelemetry.getPropagators().getTextMapPropagator();
	//Inject the current context with send span into the message
	propagator.inject(Context.current(), message, setter);
	//Publish the message to the given topic
	messageProducer.send(messageDestination, message);
} catch (Exception e) {
	sendSpan.recordException(e);                          //Span can record an exception 
	sendSpan.setStatus(StatusCode.ERROR, e.getMessage()); //Set span status as ERROR/FAILED
} finally {
	sendSpan.end(); //The span data is exported after you call end()
}	
package com.solace.samples.jms.snippets;

import com.solace.opentelemetry.javaagent.jms.SolaceJmsW3CTextMapGetter;
import com.solace.opentelemetry.javaagent.jms.SolaceJmsW3CTextMapSetter;
import io.opentelemetry.api.OpenTelemetry;
import io.opentelemetry.api.trace.Span;
import io.opentelemetry.api.trace.SpanKind;
import io.opentelemetry.api.trace.StatusCode;
import io.opentelemetry.api.trace.Tracer;
import io.opentelemetry.context.Context;
import io.opentelemetry.context.Scope;
import io.opentelemetry.context.propagation.TextMapPropagator;
import io.opentelemetry.semconv.trace.attributes.SemanticAttributes;
import io.opentelemetry.semconv.trace.attributes.SemanticAttributes.MessagingDestinationKindValues;
import io.opentelemetry.semconv.trace.attributes.SemanticAttributes.MessagingOperationValues;
import java.util.function.Consumer;
import javax.jms.Message;
import javax.jms.MessageProducer;
import javax.jms.Topic;

public class HowToImplementManualInstrumentation {

/**
* Example how to inject a tracing context in the Solace Message and generate a SEND span for the
* published message
*
* @param message            A Solace message that support tracing context propagation.
* @param messageProducer    JMS Message producer that can publish messages
* @param messageDestination message will be published to this topic
* @param openTelemetry      The entry-point to telemetry functionality for tracing, metrics and
*                           baggage.
* @param tracer             Tracer is the interface for Span creation and interaction with the
*                           in-process context.
*/
void howToCreateSpanOnMessagePublish(Message message, MessageProducer messageProducer,
Topic messageDestination, OpenTelemetry openTelemetry, Tracer tracer) {

//Create a new span with a current context as parent of this span
final Span sendSpan = tracer
	.spanBuilder("mySolacePublisherApp" + " " + MessagingOperationValues.PROCESS)
	.setSpanKind(SpanKind.CLIENT)
	// published to a topic endpoint (non temporary)
	.setAttribute(SemanticAttributes.MESSAGING_DESTINATION_KIND, MessagingDestinationKindValues.TOPIC)
	.setAttribute(SemanticAttributes.MESSAGING_TEMP_DESTINATION, false)
	//Set more attributes as needed
	//.setAttribute(...)
	//.setAttribute(...)
	.setParent(Context.current()) // set current context as parent
	.startSpan();

//set sendSpan as new current context
try (Scope scope = sendSpan.makeCurrent()) {
	final SolaceJmsW3CTextMapSetter setter = new SolaceJmsW3CTextMapSetter();
	final TextMapPropagator propagator = openTelemetry.getPropagators().getTextMapPropagator();
	//and then inject current context with send span into the message
	propagator.inject(Context.current(), message, setter);
	// message is being published to the given topic
	messageProducer.send(messageDestination, message);
} catch (Exception e) {
	sendSpan.recordException(e); //Span can record exception if any
	sendSpan.setStatus(StatusCode.ERROR, e.getMessage()); //Set span status as ERROR/FAILED
} finally {
	sendSpan.end(); //End sendSpan. Span data is exported when span.end() is called.
	}
}
}

Generating a Receive Span on Message Receive

Your consuming application can generate a receive span and then export it to the OpenTelemetry Collector. The following steps show you how to extract tracing context from a received message and generate a receive span:

  1. Use a SolaceJmsW3CTextMapGetter to extract any existing context from the received message:
    2. final SolaceJmsW3CTextMapGetter getter = new SolaceJmsW3CTextMapGetter();
final Context extractedContext = openTelemetry.getPropagators().getTextMapPropagator()
	.extract(Context.current(), message, getter);
  2. Set the extracted context as the current context with the makeCurrent() method. Next create a child span (receiveSpan in this example) and set the extracted context as the parent of that child span. Start the span using the startSpan() method:
    3. try (Scope scope = extractedContext.makeCurrent()) {
	//Create a child span and set extracted/current context as parent of this span
	final Span receiveSpan = tracer
		.spanBuilder("mySolaceReceiverApp" + " " + MessagingOperationValues.RECEIVE)
		.setSpanKind(SpanKind.CLIENT)
		//If the message was received on a non temporary queue endpoint
		.setAttribute(SemanticAttributes.MESSAGING_DESTINATION_KIND,
		MessagingDestinationKindValues.QUEUE)
		.setAttribute(SemanticAttributes.MESSAGING_TEMP_DESTINATION, false)
		//Set more attributes as needed
		//.setAttribute(...)
		//.setAttribute(...)
		.setParent(extractedContext) //Creates a parent-child relationship for a message publisher's application span 
		.startSpan();
//Try catch continues in next step...
  3. Accept and process the received message, then call the end() method on the receive span to export the span data:
    4. 	try {
		//Do something with the message in a callback function
		messageProcessor.accept(receivedMessage);
	} catch (Exception e) {
		receiveSpan.recordException(e);                          //Span can record exception if any
		receiveSpan.setStatus(StatusCode.ERROR, e.getMessage()); //and set span status as ERROR/FAILED
	} finally {
		receiveSpan.end(); //Span data is exported when you call span.end()
	}
}
package com.solace.samples.jms.snippets;

import com.solace.opentelemetry.javaagent.jms.SolaceJmsW3CTextMapGetter;
import com.solace.opentelemetry.javaagent.jms.SolaceJmsW3CTextMapSetter;
import io.opentelemetry.api.OpenTelemetry;
import io.opentelemetry.api.trace.Span;
import io.opentelemetry.api.trace.SpanKind;
import io.opentelemetry.api.trace.StatusCode;
import io.opentelemetry.api.trace.Tracer;
import io.opentelemetry.context.Context;
import io.opentelemetry.context.Scope;
import io.opentelemetry.context.propagation.TextMapPropagator;
import io.opentelemetry.semconv.trace.attributes.SemanticAttributes;
import io.opentelemetry.semconv.trace.attributes.SemanticAttributes.MessagingDestinationKindValues;
import io.opentelemetry.semconv.trace.attributes.SemanticAttributes.MessagingOperationValues;
import java.util.function.Consumer;
import javax.jms.Message;
import javax.jms.MessageProducer;
import javax.jms.Topic;

public class HowToImplementManualInstrumentation {

/**
* Example how to extract a tracing context from the Solace Message and generate a RECEIVE span
* for the received message
*
* @param receivedMessage  A Solace message.
* @param messageProcessor A callback function that user could use to process a message
* @param openTelemetry    The OpenTelemetry class is the entry point to telemetry functionality
*                         for tracing, metrics and baggage from OpenTelemetry Java SDK.
* @param tracer           OpenTelemetry Tracer is the interface from OpenTelemetry Java SDK for
*                         span creation and interaction with the in-process context.
*/
void howToCreateNewSpanOnMessageReceive(Message receivedMessage,
	Consumer<Message> messageProcessor,
	OpenTelemetry openTelemetry, Tracer tracer) {

//Extract tracing context from message, if any using the SolaceJmsW3CTextMapGetter
final SolaceJmsW3CTextMapGetter getter = new SolaceJmsW3CTextMapGetter();
final Context extractedContext = openTelemetry.getPropagators().getTextMapPropagator()
	.extract(Context.current(), receivedMessage, getter);

//Set the extracted context as current context
try (Scope scope = extractedContext.makeCurrent()) {
	//Create a child span and set extracted/current context as parent of this span
	final Span receiveSpan = tracer
	.spanBuilder("mySolaceReceiverApp" + " " + MessagingOperationValues.RECEIVE)
	.setSpanKind(SpanKind.CLIENT)
	// for the case the message was received on a non temporary queue endpoint
	.setAttribute(SemanticAttributes.MESSAGING_DESTINATION_KIND,
	MessagingDestinationKindValues.QUEUE)
	.setAttribute(SemanticAttributes.MESSAGING_TEMP_DESTINATION, false)
	//Set more attributes as needed
	//.setAttribute(...)
	//.setAttribute(...)
	// creates a parent child relationship to a message publisher's application span is any
	.setParent(extractedContext)
	// starts span
	.startSpan();

	try {
		// do something with a message in a callback
		messageProcessor.accept(receivedMessage);
	} catch (Exception e) {
		receiveSpan.recordException(e); //Span can record exception if any
		receiveSpan
		.setStatus(StatusCode.ERROR, e.getMessage()); //and set span status as ERROR/FAILED
	} finally {
		receiveSpan.end(); //End receiveSpan. Span data is exported when span.end() is called.
		}
		}
	}
	}