Configuring Container Logging

By default, the Solace PubSub+ software event broker generates log messages, based on the format defined in RFC 3164, and directs them to log files for each logging facility in /var/lib/solace/jail/logs. In addition, you can use the Solace CLI to direct each of the logging facilities to a remote syslog receiver. With the container package of the PubSub+ software event broker, you can also use configuration keys when you create the container to configure the event broker to direct the logging facilities to stdout . If you direct the logging facilities to stdout, you can modify the output format and use the container runtime logging drivers to forward the output to an alternate location, for example an external log aggregator, for additional storage, analysis, and reporting.

For more information about event broker logging, including a detailed description of the default syslog message format, see the Configuring Monitoring.

Selecting Logging Facilities

When you use configuration keys to format or direct log output, you must include the logging facility name in the configuration key name. This allows you to control the output destination and format of each facility independently.

You can configure the output destination, and format (if stdout is specified as the destination) for each of the following logging facilities:

Logging Facility Description
debug Actions performed or errors encountered by debug processes used for internal debugging.
command Actions performed or errors encountered by commands issued at the Solace Command Line Interface (CLI) prompt, by CLI script files, or by SEMP.
system Incidents or errors encountered by event broker processes that relate to the health of the event broker. The system facility is a subset of the event facility.
event Incidents or errors encountered by event broker processes, for example event broker events related to publishers and subscribers.
kernel Action performed or errors encountered by low level processes used for internal debugging.

For example:

logging_command_output=stdout
logging_system_output=all

In this example, output for the command facility is directed to stdout and output for the system facility is directed to both a file on the local host and stdout.

Configuring the Log Output Destination

To configure the software event broker logging facility output destination, include the logging/<log_facility>/output configuration key during container initialization and specify one of the following values:

Output Destination Description
file

Default output destination. Directs the logging facility to the appropriate log file in /usr/sw/jail/logs on the local host.
stdout

Directs the logging facility to stdout.
all

Directs the logging facility to both a log file on the local host and stdout.

none

Directs the logging facility to neither a log file nor to stdout (effectively NULL).

Be aware that setting the logging facility output destination to none is equivalent to disabling the logging facility.

Solace recommends against this setting because logs generated for the corresponding facility are effectively lost.

For example:

logging_debug_output=all
logging_command_output=all
logging_system_output=all
logging_event_output=all
logging_kernel_output=all

In this example, output for all logging facilities is directed to both a log file on the local host and the software event broker container's stdout I/O stream.

If you direct multiple logging facilities to stdout, all facilities are combined into a single stream. To extract the facilities from the output, use a log aggregator (Logstash, Fluentd) and filter on the logging facility field you are interested in.

Configuring the Log Output Format

To configure the logging facility output format for a container, include the logging/<log_facility>/format configuration key during container initialization and specify one of the following values:

Log Output Format Description
graylog

JSON format that is supported by Graylog, Logstash, and Fluentd.
rfc5424

Syslog format that is supported by syslog and log aggregators such as Logstash and Fluentd via plugins.
raw

No formatting is applied to the logging facility.

If you select this value, log output does not include timestamp information.
legacy

Default syslog format. This is the format that all logs are written in when you specify the file output destination.

For example:

logging_debug_format=graylog
logging_command_format=graylog
logging_system_format=graylog
logging_event_format=graylog
logging_kernel_format=graylog

In this example, logging facility output is converted to a JSON format that is supported by Graylog log aggregators such as Logstash or Fluentd.

Changing this value affects only the format of logging facilities sent to stdout. The log files in /usr/sw/jail/logs are always in the legacy format.

Forwarding the Log Output

If you configure the output destination for the software event broker container logs to stdout, you can use logging drivers to forward the combined log stream for further storage, reporting, and analysis.

For Docker or Podman, include the --log-driver <string> create option during container initialization and specify a supported logging driver, in combination with the appropriate --log-opt <value> options, to forward the log stream to your desired destination.

For example: 

--log-driver syslog
--log-opt syslog-format=rfc5424micro
--log-opt syslog-address=tcp://192.168.1.1:12201

In this example, all logging facilities directed to stdout are forwarded to an external syslog server located at 192.168.1.1:12201.

For more information about using logging drivers and corresponding options to forward the stdout I/O stream, as well as which logging drivers are supported, see: