Getting Started With the Connector for Google Pub/Sub

Presuming you’re using the default application.yml file provided within the download, follow one of the quick starts below to connect a PubSub+ event broker and a Google Cloud Pub/Sub broker . The quick starts use default credentials as examples to get started with two workflows enabled, workflow 0 and workflow 1.

Where:

  • Workflow 0 consumes messages from the Solace PubSub+ queue, Solace/Queue/0 and publishes them to the Google Cloud Pub/Sub producer destination, producer-destination.

  • Workflow 1 consumes messages from theGoogle Cloud Pub/Sub subscription, consumer destination, consumer-destination-sub and publishes them to the to the Solace PubSub+ topic, Solace/Topic/1.

A workflow is the configuration of a flow of messages from a source to a target. The connector supports up to 20 individual, concurrent workflows per instance.

The connector does not provision queues that do not exist.

Prerequisites

Quick Start Common Steps

These are the steps that are required to run all quick-start examples:

  1. Decide which Google Cloud Pub/Sub  authentication mechanism will be used and accordingly have application_default_credentials.json or the service account key JSON, etc. available for use.

  2. Update the provided samples/config/application.yml with the values for your deployment.

Quick Start: Running the Connector Using the Command Line

Run:

  java -jar  pubsubplus-connector-gcppubsub-<version>.jar --spring.config.additionallocation=file:samples/config/

Where <version> is the version of the connector you are using, such as 2.1.7.

By default, this command will detect any Spring Boot config files as per Spring Boot's default location.

For more info, see Configure Locations to Find Spring Property Files.

By default, the Google Cloud Pub/Sub client library looks for the authentication credentials as per the Application Default Credentials (ADC) Search Order. For more information, see Google Cloud Pub/Sub Connection Details.

Quick Start: Running the Connector Using the start.sh Script

For convenience, you can start the connector through the shell script using the following command:

chmod 744 ./bin/start.sh
./bin/start.sh [-n NAME] [-l FOLDER] [-p PROFILE] [-c FOLDER] [-ch HOST] [-cp PORT] [
-j FILE] [-cm] [-cmh HOST] [-cmp PORT] [-mh HOST] [-mp PORT] [-o OPTIONS] [-b]

If you use invalid parameters, such as in the following example:

./bin/start.sh -l dummy_folder -c dummy_folder -j dummy_file.jar

The script shows you all errors at the same time:

Solace PubSub+ Connector for Google Pub/Sub

Connector startup failed:

Following folder doesn't exists on your filesystem:     'dummy_folder'		
Following folder doesn't exists on your filesystem:     'dummy_folder'
Following file doesn't exists on your filesystem:       'dummy_file.jar'

In situations where you have don't provide a parameter, the script runs with the predefined values as follows:

The default name for the JAR file is pubsubplus-connector-gcppubsub-2.1.7.jar.

Parameter Default Value Description
-n, --name (only available for connectors version 2.0.0 and later) application The name of the connector instance, that is configured in [spring.application.name]. This name impacts the grouping of connectors only.

-l, --libs

./libs

The directory that contains the required and optional dependency JAR files, such as Micrometer metrics export dependencies (if configured). If this option is not specified, it will use the current ./libs/ directory.

-p, --profile

empty, no profile is used

The profile to be used with the connector’s configuration. The configuration file named 'application-<profile>.yml' is used. If  this option is not specified, no profile is used.

-c, --config

./ or current folder

The path to the folder containing the configuration files to be applied when the connector starts up for chosen profile. If not specified, the current directory is used.
-H, --host  (only available for connectors version 2.0.0 and later) 127.0.0.1 Specifies the host where the connector runs.
-P, --port (only available for connectors version 2.0.0 and later) 8090 Specifies the port where connector runs.
-mp, --mgmt_port (only available for connectors version 2.0.0 and later) 9009 Specifies the management port for back calls of current connector from PubSub+ Connector Manager. This parameter is ignored if the -cm parameter is not provided.

-j, --jar

The name of the JAR file. See JAR file for this connector.

The path to the specified JAR file to start the connector. If the option is not specified, the default JAR file is used from the current directory.
-cm, --manager (only available for connectors version 2.0.0 and later) application

Specifies Connector Managerto use the configuration storage and allows you to enable the cloud configuration for the connector. When this parameter is enabled, you can specify the -mp or --mgmt_port, -H, or --host, and -cmh with the -cmp parameters, unless you want to use the default values for those parameters.

This option disables the listed parameters to be read from the configuration file. In this case, the operator must explicitly specify the parameters for the script, otherwise default values are used.

-cmh, --cm_host  (only available for connectors version 2.0.0 and later)

 127.0.0.1 Specifies the host where Connector Manager is running. This parameter is ignored if -cm is not provided.
-cmp, --cm_port  (only available for connectors version 2.0.0 and later) 9500 Specifies the port where Connector Manager is running. This parameter is ignored if -cm is not provided.

-o, --options

No default values

Specifies the JVM options used on when the connector starts. For example, -Xms64M -Xmx1G.
-tls  (only available for connectors version 2.0.0 and later) N/A Specifies to use HTTPS instead of HTTP. When this parameter is used, the configuration file must contain an additional section with the preconfigured paths for the key store and trust store files.
-s, --show (only available for connectors version 2.0.0 and later) N/A Performs a dry run (does nothing). The output prints the start CLI command and its raw output and exits. This parameter is useful to check your parameters without running the connector.

-b, --background

N/A

Runs the connector in the background. No logs are shown and the connector continues running in detached mode.

-h, --help

N/A

Prints help information and exits.

The script also provides more help information from command line when you use the -h option. For more configuration examples of starting the connector together with Connector Manager, see the examples provided in the Connector Manager archive.

Quick Start: Running the Connector as a Container

The following steps show how to use a sample docker compose file has been packaged for your convenience:

  1. Add volume mapping in the sample docker-compose.yml for application_default_credentials.json from the local path to the /app/external/gcp/config/application_default_credentials.json target path.

    solace-pubsub-connector-gcppubsub:
  volumes:
    - type: bind 
      source: /path/to/local/gcloud/application_default_credentials.json
      target: /app/external/gcp/config/application_default_credentials.json   
      read_only: true

  2. Change to the docker directory:

    cd samples/docker

    This directory contains both the docker-compose.yml file as well as a .env file that contains environment secrets required for the container’s health-check.

  3. Run the connector:

    docker-compose up -d

    This sample docker compose file:

    • Exposes the connector’s 8090 web port to 8090 on the host.

    • Connects to a PubSub+ event broker and the Google Cloud Pub/Sub broker exposed on a host using default ports.

    • Mounts the samples/config directory.

    • Mounts the previously defined libs directory.

    • Creates a healthcheck user with read-only permissions.

      • The default username and password for this user can be found within the .env file.

      • This configuration overrides any users you have defined in your application.yml. See Security for more information.

    • Uses the connector’s management health endpoint as the container’s health check.

For more info about how to use and configure this container, see Modifying the Container Configuration.