Storage Configuration for Docker Images

The PubSub+ Docker Image has predefined mount points for an event broker's storage-elements. If you leave them unspecified in the docker create command, all storage-elements (except diagnostics) are automatically created as anonymous volumes on the Docker Host outside the backing store. For production deployments, we recommend that you assign the event broker's storage-elements to external storage devices. In this section we'll show you some examples of how to do that.

Before you begin

To begin assembling the requirements necessary for assigning your software event broker's storage-elements to external storage volumes, consult the following material:

Storage Concepts

There are 4 concepts that will help you better understand software event broker storage-elements and how to assign them to external storage.

  • Union Filesystem: Event broker Docker Containers are read-write layers in a Docker Host union filesystem.
  • Ephemeral Data: Data stored by an event broker Docker Container in the union filesystem is ephemeral.
  • Persistent Data: Data stored by an event broker Docker Container outside the union filesystem on external devices is persistent.
  • Docker Volumes: Storage for persistent data.

Union Filesystem

A union filesystem is a copy-on-write (COW) filesystem created by the Docker Engine on the Docker Host, managed by a storage-driver like overlay2 or devicemapper, for Image and Container storage. It allows for efficient storage of Container data because COW helps to save storage space by not requiring an Image to be duplicated every time a new Container is created.

When a software event broker Docker Image is loaded into a Docker Host, an associated read-only filesystem layer is created in the Docker Host's union filesystem. Then, when a software event broker Container is created, a read-write filesystem layer is added to the union filesystem to record differences in data between the Image and Container. This means that when writing to a file in the Container, the storage-driver creates a copy of the file in the Container's read-write filesystem that overrides the version associated with the Image, and the new data is written to the copy.

When a Container is destroyed, its associated read-write layer is deleted, and any modifications contained in the layer are lost. This is why data stored in the union filesystem is often referred to as ephemeral. We'll examine this concept and its implications in Ephemeral Data.

Ephemeral Data

Data stored by a software event broker Docker Container in the Union Filesystem is ephemeral.

When an event broker Docker Container is stopped, all the changes made inside the Container are confined to the read-write layer, and since the read-write layer is retained when the Container is stopped, the Container can be restarted. In contrast, when a Container is deleted, the read-write layer is also deleted, and all of the modifications made by the event broker Container are lost. Since the Container read-write layer doesn't exist after Container deletion, it's referred to as ephemeral.

Upgrading from one event broker release to another causes the Container to be deleted in the process, so one reason for using Docker Volumes for state storage instead of the union filesystem is to persist state across an upgrade.

Persistent Data

In contrast to Ephemeral Data, persistent data continues to exist after a Container's deletion. In terms of the software event broker, ephemeral data disappears when the event broker is deleted. Data crucial to the success of your deployment needs to be assigned to persistent storage that will continue to exist beyond the life of your particular software event broker.

Persistent data is stored on Docker Volumes, which are storage volumes that exist outside of the Union Filesystem - they don't get deleted when your software event broker is deleted.

Docker Volumes

A Docker Volume is storage located outside the Container execution environment that has been mounted on the Container's filesystem and bypasses the Union Filesystem. When Docker Volumes are used for state storage, then all filesystem changes made within the Container at the Docker Volume mount position are recorded directly in the underlying storage.

Docker Volumes are recommended for use as the means to create Persistent Data storage for a software event broker. Also, they can provide the event broker with better performing storage in comparison to the union filesystem.

Docker Volumes may be created via the Docker Engine client volume command as independent resources, or on-the-fly when a Docker Container is created using the Docker Engine client create command's --volume parameter.

For more information on Docker Volumes, refer to the Docker Documentation.

Example: Linux

To assign storage-elements to dedicated external volumes, do the following:

  1. Attach a disk, or disks, to the host depending on your storage-element-to-disk assignment plan. Since the specific steps for performing this task vary from one environment to the next, it’s recommended that you consult your environment’s documentation for instructions.
  2. Create the volumes. For detailed instructions, refer to the Docker documentation.
  3. Update the docker create command using the --volume option to assign the storage-elements to the volumes you've created. In this example, all the storage-elements have been re-assigned, but you can delete lines in accordance with your own plan.

    docker create \
    list of options \
    --volume <jail-volume-name>:/usr/sw/jail \
    --volume <diagnostics-volume-name>:/var/lib/solace/diags \
    --volume <spool-volume-name>:/usr/sw/internalSpool \
    --volume <adbBackup-volume-name>:/usr/sw/adb \
    --volume <adb-volume-name>:/usr/sw/internalSpool/softAdb \
    --volume <var-volume-name>:/usr/sw/var \
    list of options \

Example: Docker for Windows

To expand the default storage capability of software event broker containers in Docker for Windows, you can make use of drives from the host, but these drives must be shared with the Docker for Windows Linux VM. Shared drives are configured in the Docker Settings menu.

To assign additional directories on the host as external storage, add the following to the docker run command line when starting a software event broker:

-v <host-path>:<mount-path>

Where:

<mount-path>: is the path to the storage-element shown in the Storage-Element Properties table presented in Storage-Element Characteristics & Requirements.

For example, to assign the message spool:

> docker run -v C:\Users\username\storage\data:/usr/sw/internalSpool -d -p 8080:8080 -p 55555:55555 --shm-size=2g --env 'username_admin_globalaccesslevel=admin' --env 'username_admin_password=admin' --name=solace solace-pubsub-standard:8.10.x.x

This example mounts the internalspool volume on the host in the C:\Users\username\storage\data directory.

Example: Docker for Mac

To expand the default storage capability of software event broker containers in Docker for Mac, you can make use of drives from the host, but these drives must be shared with the Docker for Mac Linux VM. Shared drives are configured in the Docker Settings menu.

To assign additional directories on the host as external storage, add the following to the docker run command line when starting a software event broker:

-v <host-path>:<mount-path>

Where:

<mount-path>: is the path to the storage-element shown in the Storage-Element Properties table presented in Storage-Element Characteristics & Requirements.

For example:

> docker run -v /Users/username/storage/data:/usr/sw/internalSpool -d -p 8080:8080 -p 55555:55555 --shm-size=2g --env 'username_admin_globalaccesslevel=admin' --env 'username_admin_password=admin' --name=solace solace-pubsub-standard:8.10.x.x

This example mounts the internalspool volume on the host in the /Users/username/storage/data directory.

Setting max-message-spool when scaling up the message spool

If you are scaling up storage space for the message spool, after the event broker Docker Container is created you'll need to change the value of max-spool-usage with this Solace CLI command,

solace(configure)# hardware message-spool max-spool-usage <new-maximum>

It's recommended that 20% of the message spool storage device be reserved as overhead, so the minimum recommended value for <new-maximum> in the above command is 0.80 times the capacity of your external block device