Managing Storage for Container Images

By default all files created inside a container are stored on a writable container layer. This means that:

  • The data doesn't persist when that container no longer exists, and it can be difficult to get the data out of the container if another process needs it.
  • A container's writable layer is tightly coupled to the host machine where the container is running. You can't easily move the data somewhere else.
  • Writing into a container's writable layer requires a storage driver to manage the filesystem. The storage driver provides a union filesystem, using the Linux kernel. This extra abstraction reduces performance as compared to using data volumes, which write directly to the host filesystem.

    Support is provided for any storage driver—overlay2 is a good choice.

There are two options for containers to store files in the host machine so that the files are persisted even after the container stops: volumes and bind mounts.

For more information about storage options for containers, see Manage Data in Docker in the Docker documentation.

Use a Volume for the Storage-Group

The container image has a predefined mount point for an event broker's storage-group (/var/lib/solace). If you leave the storage-group unspecified when you start the container, it will be stored in the container's writable layer. This can cause the container to run out of space in the union filesystem and prevent data from being properly migrated during upgrade. We strongly recommended allocating storage outside the union filesystem for the storage-group.

The software event broker requires storage with medium to high bandwidth/IOPs and low latency. For information about getting the best storage performance, consult the best practices documentation for your platform.

A volume is storage in a part of the host filesystem that is managed by Docker and that has been mounted on the container's filesystem outside the writable layer. We recommend using volumes as the means to persist data for a software event broker container.

For the storage-group mounted as a volume, we recommend xfs as the filesystem, because it has better performance than ext4.

Use External Storage Devices in Production

For production deployments, we recommend that you assign the event broker's storage-group to an external storage device. In the container filesystem, the storage-group is stored in the directory /var/lib/solace.

For information about storage options for Kubernetes, see Storage in the Kubernetes documentation.

For information about storage options for OpenShift, see Understanding Persistent Storage in the OpenShift documentation.

The examples that follow show how to assign external storage with Docker, using both volumes and bind mounts.

Example: Docker for Linux

To assign a storage-group to a dedicated external volume, do the following:

  1. Attach a disk, or disks, to the host. Since the specific steps for performing this task vary from one environment to the next, we recommend that you consult your environment’s documentation for instructions.
  2. Create the new storage-group volume. For detailed instructions, refer to Use Volumes in the Docker documentation.

  3. Create a new container that mounts the new storage-group volume and maps it to /var/lib/solace:

    docker create --network=host --uts=host --shm-size=1g --ulimit core=-1 --ulimit memlock=-1 --ulimit nofile=2448:42192 \
    --env 'username_admin_globalaccesslevel=admin' --env 'username_admin_password=admin' --name=solace \
    --mount source=storage-group,target=/var/lib/solace  solace-pubsub-enterprise:<version>

If you are using SELinux and need to change the label of the host file or directory being mounted into the container, you must use the --volume parameter. The --mount parameter does not support the z or Z options for modifying SELinux labels. For more information, see Configure the SELinux Label in the Docker documentation.

Example: Docker for Windows

To expand the default storage capability of software event broker containers in Docker for Windows, you can make use of drives from the host, but these drives must be shared with the Docker for Windows Linux VM. Shared drives are configured in the Docker Settings menu.

To assign a directory on the host as external storage for the storage-group using a bind mount, add the following to the docker run command when you start a software event broker, replacing <host-path> with the path on the host where you want the storage-group to be mounted:

--mount type=bind,source=<host-path>,target=/var/lib/solace

The following example uses C:\storage\data as external storage for the storage-group:

> docker run --mount type=bind,source=C:\storage\data,target=/var/lib/solace -d -p 8080:8080 -p 55555:55555 ^
--shm-size=1g --env 'username_admin_globalaccesslevel=admin' --env 'username_admin_password=admin' ^
--name=solace solace-pubsub-standard:<version>

If a Windows process is using one of the ports you are requesting, the command above fails with a "Ports are not available" error. You can use a different port, or you can reserve ports so that Windows can't use them. For details, see the related article on the Solace community.

Example: Docker for Mac

To expand the default storage capability of software event broker containers in Docker for Mac, you can make use of drives from the host, but these drives must be shared with the Docker for Mac Linux VM. Shared drives are configured in the Docker Settings menu.

To assign a directory on the host as external storage for the storage-group using a bind mount, add the following to the docker run command when you start a software event broker:

--mount type=bind,source=<host-path>,target=/var/lib/solace

The following example uses /mnt/solace as external storage for the storage-group:

> docker run --mount type=bind,source=/mnt/solace,target=/var/lib/solace -d -p 8080:8080 -p 55556:55555 \
--shm-size=1g --env 'username_admin_globalaccesslevel=admin' --env 'username_admin_password=admin' \
--name=solace solace-pubsub-standard:<version>

On MacOS Big Sur and later, port 55555 (the default SMF port for the software broker) is blocked. If this port is mapped to a port on a Docker container, the container fails to start, either silently or with a "port in use" error. To avoid this problem, map the SMF port to a different port on the host.