Distributed Tracing OpenTelemetry Span Fields

As discussed in Context Propagation, message traces are made up of spans that are emitted at each point along the event mesh, consumed (and converted) by the Solace Open Telemetry Receiver, and sent to your chosen backend, which reassembles these spans and provides a visual view of the movement and processing of an event message. Each span that passes to the backend is made up of span fields and include span attributes (key-value pairs).

These tables list the OpenTelemetry span attributes in the event message that are sent to your observability backend (from the OpenTelemetry Collector with a configured Solace Open Telemetry Receiver).

For more information on traces, spans, span attributes (key-value pairs), and more, see OpenTelemetry Traces.

OpenTelemetry Specification Attributes

These are general attributes defined by OpenTelemetry that apply to spans across various types of systems, providing metadata like identification, timing, and context.

The contents in the following table are based on OpenTelemetry Specification version 1.22.0. For more information, see OpenTelemetry Specifications.

  Span Attribute Type   Description Examples Attribute present in:
Send Spans Receive Spans Move Spans Delete Spans
TraceId 16 byte array

The traceID associated with the span.

 4bf92f3577b34da6a3ce929d0e0e472a
SpanId 8 byte array

The spanID associated with the span.

b3c2a1f3d1e0a3b4
TraceState set of key-value pairs

The TraceState is a part of the span context, for more information see OpenTelemetry - TraceState.

 vendor1=value1,vendor2=value2 ...      
SpanKind enum

Specifies the role of the span within a trace. Can be PRODUCER or CONSUMER.

PRODUCER

StartTimeUnixNano integer

The start timestamp of the span.

 1693056000000000000
EndTimeUnixNano integer

The end timestamp of the span.

 1693056060000000000
Status enum

The status of the OpenTelemetry span. Can be one of: OK, ERROR, or UNSET

OK

 
ParentSpanId 8 byte array

The spanID associated with the parent span.

 b3c2a1f3d1e0a3b4

OpenTelemetry Semantic Convention Attributes

OpenTelemetry Semantic Convention Attributes are attributes with standardized naming conventions. They are used to describe common operations, resources, and events across different systems in a consistent way.

The contents in the following table are based on OpenTelemetry Semantic Conventions version 1.26.0. Any future changes made by OpenTelemetry may require corresponding updates to these attributes. For more information, see OpenTelemetry Semantic Conventions Attributes Registry.

  Span Attribute Type Description Examples Attribute present in:
Send Spans Receive Spans Move Spans Delete Spans
messaging.system string

The main messaging system of the event mesh.

 SolacePubSub+
service.instance.id string

The message VPN of the event broker.

 my-production-vpn
service.version string

The current version of the event broker.

 10.10.1
service.name string

The logical name of the current service.

 solace-cloud-client
messaging.operation.name string

The system-specific name of the messaging operation.

Can be one of: receive, send, move, or delete.

receive

messaging.operation.type string

A string identifying the type of the messaging operation. Can be one of:

  • receive—Event message received or enqueued by the event broker.
  • publish—Event message sent by the event broker to the consumer or acknowledged by the consumer.
  • move—Event message moved to a configured dead message queue.
  • delete—Event message deleted by the event broker or an administrator.

publish

network.protocol.name string

The protocol used when the message was received by the event broker.

Can be one of: SMF, REST, AMQP, or MQTT

SMF

 

    
network.protocol.version string

The version of protocol used. For example in MQTTit can be 3.1 or 5.0.

 3.1    
messaging.message.id string

The application message ID of the event broker. This is a user-defined property that allows an application to assign a unique identifier to each message.

 452a7c7c704887f1572b18fc2      
messaging.message.conversation_id string

The correlation ID of the message, usually set by the message producer and used to correlate messages within your messaging system.

 MyCorrelationID      
messaging.message.body.size integer

The size of the message body in bytes.

 1439      
messaging.message.envelope.size integer

The size of the message body and metadata in bytes.

 2738      
messaging.destination.name string

The topic or queue the message was published to.

 my/topic/destination  
messaging.source.kind string The kind of endpoint a message (or a batch) was published to. Either a queue or topicEndpoint. queue      
messaging.source.name string

The original Solace topic or queue a message (or a batch) was published to.

 my/publish/destination    
server.address string

The IP Address that the event broker received the message on.

 10.1.2.80      
server.port integer

The port that the event broker received the message on.

 8080      
network.peer.address string

The IP Address of the publishing client.

 10.1.2.80      
network.peer.port integer

The port of the publishing client.

 65123      

Solace Span Attributes

Solace span attributes provide additional context and details specific to message flows using Solace systems and applications.

The contents in the following table are accurate as of PubSub+ event broker, version 10.10.1 or later.

  Span Attribute Type Description Examples Attribute present in:
Send Spans Receive Spans Move Spans Delete Spans
messaging.solace.partition_number integer

The partition number of a queue or a topic endpoint within the event broker that is used to distribute messages across multiple partitions.

 3  
messaging.solace.source.
partition_number 		integer The partition number of queue or topic endpoint the message is being moved from, if that queue is a partitioned queue. 3      
messaging.solace.destination.
partition_number 		integer The partition number of the destination dead message queue (DMQ) the message is being moved to, if that queue is a partitioned queue. 5      
messaging.solace.source.kind enum

The type of the messaging source. Can be a queue or topicEndpoint.

queue

     
messaging.solace.client_username string

The client username used by a client application to authenticate and connect to an event broker.

 user_admin    
messaging.solace.client_name string

The client name is a unique identifier for each client connection to the event broker.

 client1234    
messaging.solace.operation.reason enum

The reason for deleting the message or moving it to a dead message queue. Can be one of the following:

ttlExpired—Expired time-to-live on a message.

rejectedNack— A negative acknowledgment with Rejected outcome.

maxRedeliveriesExceeded— Max redelivery reached or exceeded.

hopCountExceeded—Hop count exceeded (delete spans only).

ingressSelector— Destination unable to match any ingress selector rule (delete spans only).

adminAction— Message deleted by an administrator (delete spans only).

ttlExpired

   
messaging.solace.
replication_group_message_id 		string

The replication group message ID of the received message, used to uniquely identify a message within a replication group.

 0xA5F7D4E2B3C1A4F0    
messaging.solace.priority integer

The priority of the published message that allows you to control the order in which messages are delivered from a queue.

 9      
messaging.solace.ttl integer

The time-to-live (TTL) of the published message in milliseconds.

 60000    
messaging.solace.dmq_eligible boolean

Indicates that the dead message queue (DMQ) eligible bit was set in the message.

 true      
messaging.solace.
dropped_enqueue_events_success 		boolean

Indicates that the enqueueing of a dropped message succeeded.

 true      
messaging.solace.
dropped_enqueue_events_failed 		boolean

Indicates that the enqueueing of a dropped message failed.

 true      
messaging.solace.reply_to_topic string

The reply-to topic of the received message.

 replyto/destination      
messaging.solace.
broker_receive_time_unix_nano 		integer

Indicates when the event broker first identifies that a message is to be traced.

 1693056060000000000      
messaging.solace.
dropped_application_message_properties 		boolean

If true, indicates that not all user properties in the message were captured in the user_properties because the limit was exceeded. The event broker supports up to a total of 8 KiB of user property data.

 true      
messaging.solace.delivery_mode enum

The delivery mode of the message, can be persistent, non-persistent, or direct.

persistent

     
messaging.solace.message_replayed boolean

Indicates whether the message has been replayed by the event broker.

 true      
messaging.solace.send.outcome enum

Displays the outcome of the sent message. Can be one of the following:

  • accepted
  • rejected
  • released
  • delivery failed
  • flow unbound
  • transaction commit
  • transaction commit failed
  • transaction rollback

accepted

     
messaging.solace.user_properties.<...> string

A custom span attribute, where <...> is a user assigned key.

 some-user-value-1      
messaging.solace.destination.type enum

The type of the message destination. Can be a queue or topicEndpoint.

topicEndpoint

 
messaging.solace.enqueue_error_message string

An error message that describes why an enqueue operation failed.

"The specified queue does not exist"

     
messaging.solace.rejects_all_enqueues boolean

Indicates that all other non-error enqueue events in this span are rejected. In other words, the message is not enqueued to any destination, regardless of what some enqueue events may indicate.

 true      
messaging.solace.ttl_override boolean

TTL Override is used to set or override the TTL value for messages, regardless of the TTL specified by the message producer.

 true      
Transaction Attributes
messaging.solace.transaction_initiator string

Identifies if the ADMIN operation was performed using the management interface or by the CLIENT.

CLIENT  
messaging.solace.transaction_id string

The transaction ID. All messages that are part of the same transaction have the same value for this ID.

tx12345-abcde-67890  
messaging.solace.transacted_session_name string

The name of the local ID of the transaction event.

 OrderProcessSession  
messaging.solace.transacted_session_id string

The type of the local ID of the transaction event.

 session-98765-zyxwv  
messaging.solace.
transaction_error_message 		string

Describes why a transaction operation failed.

"Transaction timeout due to network delay"

 
messaging.solace.transaction_xid string

The transaction ID of an XA transaction

 xid-45678-pqrst-12345  
Baggage Propagation
messaging.solace.message.baggage.
<BAGGAGE_KEY NAME> 		string

Provides the value associated with the Baggage Key Name that is provided through context propagation.

 region=us-west      
messaging.solace.message.
baggage_metadata.<BAGGAGE_KEY NAME> 		string

Provides the metadata value associated with the Baggage Key Name that is provided through context propagation.

 log=true      

Deprecated Attributes

The following fields have been deprecated as of PubSub+ event broker, version 10.10.1:

  Deprecated Attribute Replacement Attribute
messaging.protocol network.protocol.name
messaging.protocol_version network.protocol.version
messaging.message_id messaging.message.id
messaging.conversation_id messaging.message.conversation_id
messaging.message.payload_size_bytes
  • messaging.message.body.size
  • messaging.message.envelope.size
net.host.ip server.address
net.host.port server.port
net.peer.ip network.peer.address
net.peer.port network.peer.port
messaging.operation
  • messaging.operation.name
  • messaging.operation.type
messaging.destination messaging.destination.name
messaging.solace.destination_type messaging.solace.destination.type