Context Propagation for Distributed Tracing

Distributed tracing allows your enterprise applications to trace the flow of messages as they travel from your publisher, through the event mesh and to the receiving application. For a detailed overview, see Distributed Tracing . For information about version requirements, see Distributed Tracing Version Compatibility.

For information about instrumenting your code for distributed tracing, see:

  • The OpenTelemetry Integration API libraries support W3C propagators only.
  • For information about configuring OpenTelemetry SDK environment variables, see OpenTelemetry SDK Configuration.
  • By default, traces include command line parameters visible to backend applications like Jaeger. It is important to disable this feature for security purposes because these parameters may contain sensitive information such as your user name and password. For instructions, see Disabling Automatic Resource Providers in the OpenTelemetry documentation in GitHub.

Instrumenting Node.js for Distributed Tracing

Manual instrumentation involves making changes to your enterprise application's source code, and allows you to inject and extract additional context, such as baggage and trace states, into messages. Context propagation makes it easy to debug and optimize your application. For more information about context propagation in Solace event messages, see Distributed Tracing Context Propagation. The following examples show you how to create spans using the OpenTelemetry API.

Manual context propagation for distributed tracing is not available in the Solace Messaging API for JavaScript.

Understanding How Context Propagation Enables Distributed Tracing in the Solace Node.js API

In your client application, you can use the OpenTelemetry API to create a span, which contains metadata about an operation in a distributed system. This span is associated with a context, which includes a unique TraceID. Next, when you use a Solace message producer to publish a message, the Solace OTel integration package injects the context, which contains the TraceID, into the message. As the message travels through the event broker and is received by a consuming application, spans are generated at each step and have the same TraceID present in the original message context. When each span is closed in the publishing or consuming application, the JavaScript OpenTelemetry API sends it to an OpenTelemetry collector, which collects, processes and exports the spans to a backend application that correlates the spans using their unique TraceID. A backend application uses the correlated spans to create a trace, which is an end-to-end snapshot detailing how the message traveled through the distributed system. If you do not use context propagation, then backend applications cannot use a unique TraceID to link the spans, making it difficult to trace the flow of messages through the distributed system.

Dependencies

To enable context propagation for distributed tracing, you must first add the OpenTelemetry Integration for Solace Node.js API artifact to your application. For OpenTelemetry version compatibility see Distributed Tracing Version Compatibility. Adding the libraries gives you access to the following two classes:

  • SolaceW3CTextMapSetter
    • use with the OpenTelemetry API's W3CTraceContextPropagator to inject context into a Message.
    • use with the OpenTelemetry API's W3CBaggagePropagatorto inject baggage into a Message.
  • SolaceW3CTextMapGetter
    • use with the OpenTelemetry API's W3CTraceContextPropagator to extract context from a Message.
    • use with the OpenTelemetry API's W3CBaggagePropagatorto extract baggage from a Message.

This guide assumes you are familiar with configuring an instance of the OpenTelemetry class. For instructions for configuring OpenTelemetry objects, see OpenTelemetry Manual Instrumentation in JavaScript in the OpenTelemetry documentation.

To use context propagation in the Solace Node.js API, include the following modules in your application:

import {
    SemanticAttributes,
    MessagingOperationValues,
    MessagingDestinationKindValues,
} from '@opentelemetry/semantic-conventions';

const opentelemetry = require('@opentelemetry/api');
const { CompositePropagator, W3CTraceContextPropagator, W3CBaggagePropagator, TraceState } = require("@opentelemetry/core");
const { BasicTracerProvider } = require("@opentelemetry/sdk-trace-base");
const { context, propagation, trace } = require("@opentelemetry/api");
const { SemanticAttributes, MessagingDestinationKindValues } = require("@opentelemetry/semantic-conventions");
const { SemanticResourceAttributes } = require('@opentelemetry/semantic-conventions');
const { SolaceW3CTextMapSetter, SolaceW3CTextMapGetter } = require("pubsubplus-opentelemetry-js-integration");

const api = require('@opentelemetry/api'); 
var solace = require('solclientjs').debug; // with logging supported

Injecting Context into an Outbound Message

Your publishing application can generate a send span and export it to the OpenTelemetry collector. The following steps show you how to inject context into a message and generate a send span for a published message:

  1. Create propagator instances. The Solace Node.js API supports trace context and baggage propagation. If you plan to use both in your application, we recommend you use a CompositePropagator which takes in an array of propagators:
    const compositePropagator = new CompositePropagator({
       propagators: [
         new W3CBaggagePropagator(),
         new W3CTraceContextPropagator(),
       ],
    });
    
  2. Set the global propagator with a W3CTraceContextPropagator() and optional W3cCBaggagePropagator that ensures all context propagation in your application follows the same rules and mechanisms. In the example below, both are contained in the compositePropagator created in step 1. Next create a BasicTracerProvider which allows your application to create and activate traces:
  3. propagation.setGlobalPropagator(compositePropagator);
    const tracerProvider = new BasicTracerProvider();
  4. Use the register() function to register the tracerProvider created in step 1 with the OpenTelemetry API. This lets you use tracer instances to create spans and record trace data. Next create a tracer. The parameter you pass to the getTracer() function represents the name of the tracer, which is used to associate the tracer with a specific component, service, or module within your application:

  5. tracerProvider.register();
    const tracer = opentelemetry.trace.getTracer("solace-pubsub-publisher-test");
    
  6. Use the OpenTelemetry API to retrieve the currently active context that will be propagated. Then, create a "publish span" and start it with the startSpan() function:

    let ctx = api.context.active();
    const span = tracer.startSpan(topicName+' send',{kind: opentelemetry.SpanKind.CLIENT}, ctx);     
  7. Set span attributes using setAttribute() function. This lets you attach additional context and meta-data to a span. You can also use setAttribute() to label the span with a descriptive name that indicates the type of operation the span represents.

    span.setAttribute('attributeKey','attributeValue');
    span.setAttribute(SemanticAttributes.MESSAGING_OPERATION, 'send');
    
  8. (Optional) Create baggage with the createBaggage() function and set it on the context by calling the setBaggage() function on your propagation instance:

    const baggage = propagation.createBaggage({
      "baggageKey": {
        value: 'baggageValue',
        metadata: undefined,
      },
    }); 
    ctx = propagation.setBaggage(ctx, baggage);        
  9. Create an instance of SolaceContextSetter and use it with a W3C context propagator to inject context into a Solace message. Send the message and set the SpanStatusCode with the setStatus() function:

    var setter = new SolaceContextSetter();
    opentelemetry.propagation.inject(ctx,solaceMessage,setter)
    try {
        solaceSession.send(solaceMessage);
        span.setStatus({
            code: opentelemetry.SpanStatusCode.OK,
            message: 'Message Sent'
         });
    } catch (error) {
        span.setStatus({
            code: opentelemetry.SpanStatusCode.ERROR,
            message: error.toString()
        });
    }                
  10. Call the end() function on the span to close it and export the data.
    span.end();        

Extracting Context from an Inbound Message

Your consuming application can generate a process span and then export it to the OpenTelemetry collector. The following steps show you how to extract tracing context from a received message and generate a process span:

  1. Use the OpenTelemetry API to create a tracer: The parameter you pass to the getTracer() function represents the name of the tracer, which is used to associate the tracer with a specific component, service, or module within your application:
  2. const tracer = opentelemetry.trace.getTracer('solace-pubsub-receiver-test');
    
  3. In your event listener for messages, create a SolaceContextGetter. Extract the trace and context information from a received message with the extract() function. Use this extracted context to create a parentContext for tracing subsequent operations:
  4. messageConsumer.on(solace.MessageConsumerEventName.MESSAGE, function (message) {
        var getter = new SolaceContextGetter();
        const parentContext = propagation.extract(opentelemetry.ROOT_CONTEXT,message,getter);
    
  5. Create a new process span and link it to the parentContext if it exists:
    if(!parentContext)
        var span = tracer.startSpan(MessagingOperationValues.PROCESS,{kind: opentelemetry.SpanKind.CLIENT});
    else
        var span = tracer.startSpan(MessagingOperationValues.PROCESS,{kind: opentelemetry.SpanKind.CLIENT},parentContext);        
  6. Set span attributes using setAttribute() function. This lets you attach additional context and meta-data to a span. You can also use setAttribute() to label the span with a descriptive name that indicates the type of operation the span represents:
    span.setAttribute('attributeKey','attributeValue');
    span.setAttribute(SemanticAttributes.MESSAGING_OPERATION, 'process');
    
  7. (Optional) Extract baggage from the parent context using the getBaggage() function:
  8. const baggage = propagation.getBaggage(parentContext);    
  9. Process the message and set and set the SpanStatusCode with the setStatus() function:
        try {
            // Process the message here
            span.setStatus({
                code: opentelemetry.SpanStatusCode.OK,
                message: 'Message processed'
            });        
            message.acknowledge();
        }  catch (error) {
            span.setStatus({
                code: opentelemetry.SpanStatusCode.ERROR,
                message: error.toString()
            });
        }
    
  10. Call the end() function on the span to close it and export the data:
    span.end();