Managing TLS Certificates for Event Broker Services in Customer-Controlled Containers

When you deploy an event broker service to a Customer-Controlled Container, you are responsible for providing and managing the TLS server certificate. Unlike event broker services deployed to Kubernetes environments, Solace does not provision or manage certificates for event broker services in Customer-Controlled Containers.

This topic covers:

Certificate Requirements

Solace never receives or stores your private key. The certificate and key remain on your infrastructure at all times.

Your TLS certificate must meet the following requirements:

  • The certificate must be valid for the DNS name or IP address that clients use to connect to the event broker service.

  • The certificate file must be in PEM format. Solace Cloud does not support DER-encoded certificates.

  • The certificate.pem file must contain all of the following in a single file: your server certificate, any intermediate CA certificates (the full chain), and the private key.

  • The private key must be an RSA key in PEM format. Solace Cloud does not support ECDSA keys. You can encrypt keys with a passphrase or leave them unencrypted.

The following table describes the supported certificate configurations:

Configuration Certificate Type Use Case

DNS name and Certificate Authority (CA)-issued certificate

A certificate signed by a public or private CA.

Production deployments where clients trust a well-known CA

IP address and self-signed certificate

Self-signed certificate with the IP address in the Subject Alternative Name (SAN) field

Development or testing environments; requires clients to explicitly trust the certificate

Installing the Certificate when Creating an Event Broker Service in a Customer-Controlled Container

When you first create an event broker service in a Customer-Controlled Container, you must place your certificate files in the designated folder before starting the container. The event broker service serves your certificate from first boot. Because no plaintext ports are published and the Mission Control Agent disables plaintext SEMP during initialization, there is no window where the broker is accessible over an unencrypted connection.

To install a TLS certificate when creating an event broker service in a Customer-Controlled Container, perform these steps:

  1. Download the deployment artifact and extract it to your deployment directory.

  2. In the extracted directory, locate the certs folder.

  3. Place your certificate and private key in the certs folder using the following file names:

    • certificate.pem: Your server certificate, any intermediate certificates (full chain), and private key, concatenated in PEM format.

    • passphrase: A text file containing the passphrase for your private key. If your private key is not encrypted, create an empty file with this name.

  4. Deploy the event broker service by running:

    docker compose up -d

At first boot, the event broker service loads the certificate from certs/certificate.pem itself. The cert-check container validates that the file exists and contains both a certificate block and a private key block before the event broker service starts. If the certificate file is missing, empty, or missing a required PEM block, cert-check fails with a clear error message and the event broker service does not start. If the passphrase is wrong or the private key doesn't match the certificate, the event broker service exits with no message in the docker compose output.

Renewing or Replacing a Certificate for an Event Broker Service in a Customer-Controlled Container

You can update the certificate for your event broker service created in a Customer-Controlled Container at any time. To renew or replace a TLS certificate, perform these steps:

  1. Obtain your new certificate and private key.

  2. Replace the files in the certs folder with your new certificate files:

    • certificate.pem: Your new server certificate, certificate chain, and private key.

    • passphrase: A text file containing the passphrase for your private key. If your private key is not encrypted, create an empty file with this name.

  3. Edit the init-config.env file and set the following variable:

    UPDATE_CERTIFICATES=true
  4. Apply the new certificate by running:

    docker compose up -d

When the container starts with UPDATE_CERTIFICATES=true, the initialization process applies the new certificate to the running event broker service using SEMP on the internal Compose network. After the certificate is updated, set UPDATE_CERTIFICATES back to false. If you leave it set to true, every subsequent docker compose up -d applies the certificate again.

Troubleshooting Certificate Issues

The following table describes common certificate issues and their solutions.

Issue Cause Solution

Initialization fails with no TLS certificate found at certs/certificate.pem error

The certificate.pem file is missing from the certs folder.

Ensure the certificate.pem file is present in the certs folder before starting the container.

Initialization fails with has no private key or has no certificate error

The certificate file is missing a required PEM block, or the file is in DER format instead of PEM.

Use PEM format and add the certificate, chain, and private key to certificate.pem.

Clients cannot connect with TLS errors

The certificate does not match the hostname or IP address clients are using to connect, or clients do not trust the certificate authority.

Make sure that the SAN of the certificate contains the address that clients use. For self-signed certificates, configure clients to trust the certificate.

The broker container keeps restarting or exits with code 2 right after start

The passphrase is wrong or missing for an encrypted key, or the private key does not match the certificate.

Fix the certs/passphrase file or the key pair. If the key is not encrypted, the passphrase file must be empty.

Initialization fails with private key error

The private key uses an unsupported algorithm (such as ECDSA) or is in DER format instead of PEM format.

Ensure the private key is an RSA key in PEM format. Convert ECDSA keys to RSA or regenerate the certificate with an RSA key.

Certificate renewal does not take effect

The UPDATE_CERTIFICATES variable was not set to true in init-config.env before redeploying.

Set UPDATE_CERTIFICATES=true in the init-config.env file and run docker compose up -d.

Certificate renewal fails with the event broker rejected the certificate (HTTP 400) and Invalid pass phrase. or Invalid private key for this certificate.

The passphrase is wrong or missing for an encrypted key, or the private key does not match the certificate.

Fix the certs/passphrase file or the key pair, then run docker compose up -d again.

Monitoring stops collecting metrics after broker restart

Only the event broker service container was restarted. The Insights Agent shares the broker's network namespace and loses connectivity when only the broker is restarted.

Run docker compose restart datadog to restart the Insights Agent.

To verify that your certificate is correctly installed on your event broker service, you can use openssl to check the certificate being served by the event broker service:

openssl s_client -connect <address>:55443 -servername <dns-name> -showcerts

This command displays the certificate chain being served by the event broker service, allowing you to verify the certificate details and expiration date. The -servername option is required when connecting by DNS name. After a certificate renewal, the SMF over TLS port (55443) uses the new certificate immediately, but the SEMP over TLS port (host port 943) may continue to serve the old certificate for up to 60 seconds.