Setting Up Event Management Agents to Upload Scanned Files

Upload Scan File mode uses two Event Management Agents; one is connected to Event Portal and the other is connected to one or more runtime event brokers and, optionally, a Confluent Schema Registry. The agent connected to your event brokers runs the discovery scan. You upload the scan file to Event Portal manually by copying it to the connected agent.

You may need to use Upload Scan File mode if you want to import runtime event broker data into additional tools that support AsyncAPI specifications, if your organization's policies don't allow you to connect Event Portal to your event brokers, or if you want to redact sensitive data from scan files before manually uploading the data to Event Portal.

The Event Management Agent files are also available in the Solace Products GitHub repository. Instructions to install an Event Management Agent and use it to scan event brokers are available in the GitHub repository. Solace recommends using Docker to install and use an Event Management Agent with Event Portal.

If you use standalone Event Management Agents, you can initiate scans and manage the data using REST APIs.

Prerequisites

For network performance, Solace recommends setting up Event Management Agents in proximity to the runtime event brokers that they scan. The computer where you set up Event Management Agents must meet these requirements:

A computer with at least 1 CPU and 1 GB of RAM
Docker

To use Upload Scan File mode, you install two Event Management Agents on two separate computers:
- One requires network access to the event brokers that you want to discover runtime data from, and to any Confluent Schema Registries that you want to get schema data from
- One requires network access to Event Portal,

Setting Up the Standalone Event Management Agent

To set up the standalone Event Management Agent, perform these steps:

Set up an Event Management Agent connection to get the connection details for the event brokers.
Pull the Event Management Agent image from Docker Hub using the following command:
```
docker pull solace/event-management-agent
```
Download the Event Management Agent connection file for the standalone agent that connects to your runtime event brokers. Make sure that your filename doesn't contain spaces or special characters or that it is wrapped in quotation marks to avoid OS-related filename errors.
Start the agent using the Docker run command in your OS terminal. Copy the command generated by Event Portal and update the following parameters:
- Replace /path/to/file/AcmeRetail.yml with the path to your connection file.
- Replace each instance of [NAME_OF_THE_EP_EVENT_BROKER]_[AUTHENTICATION_TYPE]_PASSWORD with the password required by the authentication method for each event broker that the Event Management Agent connects to.
- If included, replace SCHEMA_REGISTRY_PASSWORD with the password to connect to the Confluent Schema Registry.
```
docker run -d -p 8180:8180 -v /path/to/file/AcmeRetail.yml:/config/ema.yml --env 
EVENT_BROKER_AUTHENTICATION_PASSWORD --env SCHEMA_REGISTRY_PASSWORD --name 
event-management-agent solace/event-management-agent:latest
```
If you are following the instructions in the Cloud Console, click Next.

Setting Up the Connected Event Management Agent

To set up the connected Event Management Agent to upload scanned files, perform these steps:

Set up an Event Management Agent connection to get the connection details for the event brokers.
Pull the Event Management Agent image from Docker Hub using the following command:
```
docker pull solace/event-management-agent
```
Download the Event Management Agent connection file for the connected agent. Make sure that your filename doesn't contain spaces or special characters or that it is wrapped in quotation marks to avoid OS-related filename errors.
Run the agent installation command in your OS terminal with the following paramaters:
- Replace /path/to/file/AcmeRetail.yml with the path to your connection file.
```
docker run -d -p 8180:8180 -v /path/to/file/AcmeRetail.yml:/config/ema.yml 
--name event-management-agent solace/event-management-agent:latest
```
If you are following the instructions in the Cloud Console, click Complete Installation Setup.

The Event Management Agent takes a couple of minutes to start. When it is running, you can run discovery scans using the Event Management Agent REST APIs.

Running a Discovery Scan Using Upload Scan File Mode

If you set up an Event Management Agent as a standalone tool, you can initiate a scan using the Event Management Agent REST API. For more information about the Event Management Agent REST API, see the REST Interface documentation in GitHub.

To run a discovery scan, collect the data, and then upload it into Event Portal, perform these steps:

Using the REST API, provide the following details and run the scan:
- the id of an event broker you specified in the connection file (for example: mykafkacluster)
- the scan type (for example: KAFKA_ALL)
- the destination for the scan results (FILE_WRITER)
```
curl -X 'POST' \
	'http://localhost:8180/api/v2/ema/resources/mykafkacluster/scan' \
	 -H 'accept: */*' \
	 -H 'Content-Type: application/json' \
	 -d '{
	 "scanTypes": [
 	   "KAFKA_ALL"
	 ],
	 "destinations": [
	    "FILE_WRITER"
	 ]
}'
```
Export the scan file as a zip file that contains the discovered data in JSON format, using the following command. Specify these details:
- Replace SCAN_ID with the ID of the discovery scan
- Replace SCAN_FILE.zip with the name of the zip file
```
curl -X 'GET' \
	 'http://localhost:8180/api/v2/ema/resources/export/SCAN_ID/zip' \
	 -H 'accept: */*' \
	 --output SCAN_FILE.zip		
```
Transfer the zip file using a method that conforms to your organization's security policies from the computer with the standalone Event Management Agent to the computer with the Event Management Agent connected to Event Portal.

Upload the zip file to the connected Event Management Agent using the following curl command, providing the name of the zip file.

curl -X 'POST' \
	'http://localhost:8180/api/v2/ema/resources/import' \
	 -H 'accept: */*' \
	 -H 'Content-Type: multipart/form-data' \
	 -F 'file=@SCAN_FILE.zip;type=application/zip'

After you upload the file to the Event Management Agent, the scanned data is automatically uploaded to Event Portal.

Scan Type

The scanTypes parameter specifies what the Event Management Agent scans. It has the following values:

Solace

SOLACE_QUEUE_CONFIG: Solace queue configurations
SOLACE_QUEUE_LISTING: Solace queue listing
SOLACE_SUBSCRIPTION_CONFIG: Solace subscription configurations
SOLACE_ALL: All of the above

Kafka

KAFKA_BROKER_CONFIGURATION: Kafka broker configurations
KAFKA_CLUSTER_CONFIGURATION: Kafka cluster configurations
KAFKA_CONSUMER_GROUPS: Kafka consumer groups
KAFKA_CONSUMER_GROUPS_CONFIGURATION: Kafka consumer group configurations
KAFKA_FEATURES: Kafka feature
KAFKA_PRODUCERS: Kafka producers
KAFKA_TOPIC_CONFIGURATION: Kafka topic configurations
KAFKA_TOPIC_CONFIGURATION_FULL: Kafka topic configurations full
KAFKA_TOPIC_LISTING: Kafka topic listing
KAFKA_TOPIC_OVERRIDE_CONFIGURATION: Kafka topic override configurations
KAFKA_ALL: All of the above

Confluent Schema Registry:

CONFLUENT_SCHEMA_REGISTRY_SCHEMA: Confluent Registry schemas

Viewing the Event Management Agent Logs

The Event Management Agent logs can help you monitor Event Management Agent output and know when the Event Management Agent is ready. You can get the logs from Docker using the following command.

docker logs -f event-management-agent

Provide feedback