Gathering Diagnostics from Software Event Broker Containers

A complete set of event broker diagnostic files and debug logs are sometimes required for Solace Customer Support to determine the root cause of an operational problem. To assist with the investigation, you can use either of the following to gather diagnostic information about a Solace PubSub+ software event broker container:

  • gather-diagnostics-host utility from the container
  • gather-diagnostics command from the Solace CLI

Gather-Diagnostics-Host Utility

We recommend that you use the gather-diagnostics-host utility instead of the gather-diagnostics CLI command. The utility can be run even if the software event broker container isn’t running, Solace CLI is not accessible, or there is a suspected misconfiguration of the container host.

To use the gather-diagnostics-host utility, you must have a Linux host running Python 2.7 or greater.

The commands below assume that you are running your container in rootful mode. If you are running in rootless mode, you may not need to use sudo.

The gather-diagnostics-host utility is delivered in the software event broker container file system. To extract the utility, use the docker cp or podman cp command.

>sudo docker cp <CONTAINER_NAME>:/usr/share/solace/gather-diagnostics-host <DESTINATION_PATH>

For example:

>sudo docker cp solace:/usr/share/solace/gather-diagnostics-host /usr/local/bin

The gather-diagnostics-host utility attempts to gather core dump files from:

  1. Any container host path mounted as a volume in the software event broker container at /var/lib/solace/diags.
  2. The container host path specified in the Linux kernel core_pattern if that path is specified as an absolute path. That is, relative paths and pipes to user space programs will not be interpreted. For more information, see Managing Core Files.
  3. A user-specified path provided as a gather-diagnostics-host command line parameter.

gather-diagnostics-host collects diagnostics information from software event broker containers. When run without additional command line parameters, it will gather information from all containers created from the default container image named solapp:currentload or, if none exist, containers using the default name solace. The user may specify a container, by name, using --name, or a specific container image, using --image.

To run the gather-diagnostics-host utility, enter the following command in the Linux host environment:

gather-diagnostics-host [-h] [-f <filename>] [-d <days>] [-n <container>] [-i <repository>:<tag>] [--no-container] [--core-host-path pattern] [-V] [-D]

Where:

-h or --help displays the gather diagnostics help information.

-f filename or --file filename outputs a single compressed file of diagnostics data. A default file name will be used if one is not specified.

-d days or --days days specifies the number of days of history to capture.

-n container or --name container specifies a particular container name or ID to gather diagnostic data for. If no container is specified, all containers matching the container image will be used.

-i repository:tag or --image repository:tag specifies the container image to use to find matching containers. The default value is solace-app:currentload.

--no-container specifies do not attempt to run gather-diagnostics-host inside the container.

--core-host-path pattern specifies the path to core files to be used as regex. This script will capture all files on the host matching pattern.

-V specifies to increase output verbosity.

-D specifies debug mode, which describes utility actions without doing anything.

Using Gather-Diagnostics-Host on Docker for Windows

For convenience, the gather-diagnostics-host support utility is bundled in the software event broker container image. To use this utility in Docker for Windows, create a new sidecar container with privileged settings based on the software event broker container image with a different ENTRYPOINT command (gather-diagnostics-host).

For example: 

> docker run --rm -t --privileged -u root --volumes-from solace -v c:\Users\username\storage\diags:/tmp -v /var/run/docker.sock:/var/run/docker.sock solace-pubsub-standard:<version> /usr/share/solace/gather-diagnostics-host --sidecar

This example creates a sidecar container which runs the gather-diagnostic-host utility on the running container named solace and places the output in a compressed tar file in the C:\Users\username\storage\diags directory.

By default, the gather-diagnostics-host utility places the /tmp directory inside the new container and removes it when the container is removed. Solace recommends mounting the /tmp directory to a persistent location on a shared disk as demonstrated in the example above.

Using Gather-Diagnostics-Host Utility on Docker for Mac

For convenience, the gather-diagnostics-host support utility is bundled in the software event broker container image. To use this utility in Docker for Mac, create a new sidecar container with privileged settings based on the container image with a different ENTRYPOINT command (gather-diagnostics-host).

For example:

> docker run --rm -t --privileged -u root --volumes-from solace -v /Users/username/storage/diags:/tmp -v /var/run/docker.sock:/var/run/docker.sock solace-pubsub-standard:<version> /usr/share/solace/gather-diagnostics-host --sidecar

This example creates a sidecar container which runs the gather-diagnostic-host utility on the running container named solace and places the output in a compressed tar file in the Users/username/storage/diags directory.

By default, the gather-diagnostics-host utility places the /tmp directory inside the new container and removes it when the container is removed. Solace recommends mounting the /tmp directory to a persistent location on a shared disk as demonstrated in the example above.

Using the Gather-Diagnostics CLI Command

You can use the Solace CLI gather-diagnostics command to gather diagnostics for a software event broker container if you don't have access to the container host; that is, if you only have Solace CLI access to the software event broker container.

The gather diagnostics command automatically gathers and packages the information. When the command completes, the retrieved files and logs are saved as a single compressed file, and the name of this diagnostics file and its location are printed to the screen.

A File Transfer User Account is required to access the generated diagnostic file. For more information about account creation see Configuring File Transfer User Accounts

Use a third-party application, such as Filezilla or WinSCP (Windows), or command-line utilities such as SCP or FTP to access the compressed files and move them to a local directory. The application will ask for a username and password, enter the ones that you created when making your file transfer account.

To gather event broker diagnostics, enter the following commands:

solace> enable 
solace# admin
solace(admin)# gather-diagnostics [days-of-history <days>]

Where:

days-of-history <days> specifies the number of days to gather diagnostics data for. By default, diagnostic data is gathered for one day.