Discovering Runtime Data

Runtime Event Manager enables you to discover runtime data from your event brokers and upload the collected data into Event Portal. You can then use the runtime data to compare your runtime event brokers with a modeled event mesh and add or update objects in Designer that represent the applications and events in your event flows. For more information, see Auditing and Importing Runtime Data.

The following diagram illustrates the components involved in discovering runtime data:

Component	Description
Event Broker (runtime)	An event broker mediates the communication of event messages between producers and consumers in an event mesh. When an application publishes an event, the message is sent to an event broker, which then routes it on to subscribing clients. An event broker can be a hardware appliance, software, or software-as-a-service (SaaS). To discover runtime data from an event broker, you create a model event broker that represents the runtime event broker in a modeled event mesh and install an Event Management Agent to create a connection between the runtime event broker and model event broker in Event Portal. Third-party event-driven architecture products may use other terms such at broker, message broker, or messaging service.
Schema Registry (runtime)	A schema registry is a centralized repository for managing and validating schemas for Kafka topic producers and consumers. You can connect a Confluent Schema registry to an Event Management Agent to discover and import Kafka subjects into Event Portal.
Event Management Agent	The Event Management Agent is a tool that discovers event and configuration data related to the event streams flowing through the event brokers, and then uploads the data to Event Portal. You connect the Event Management Agent to runtime event brokers and to the corresponding model event brokers in Runtime Event Manager. The Event Management Agent can then run discovery scans on runtime event brokers and upload the collected data to Event Portal. For Kafka event flows, an Event Management Agent can also connect to a Confluent Schema Registry to discover subjects related to Kafka topics.
Model Event Broker	In Event Portal, a model event broker represents an actual event broker in an event mesh or the gateway node in an event broker cluster. You can add multiple model event brokers to a Solace modeled event mesh to represent the actual event brokers that are part of your event mesh. A Kafka modeled event mesh must include one model event broker to represent the Kafka cluster in an event flow. When you associate an application in Designer with a model event broker in Runtime Event Manager, event flows for the application represent topic subscriptions and queues or consumer groups on the runtime event broker or cluster. When you set up an Event Management Agent to connect a model event broker to the runtime event broker that it represents, you can collect runtime data and audit the data against the model and then update the runtime or the model to keep them in sync.
Modeled Event Mesh	A modeled event mesh represents an actual event mesh with multiple event brokers or a Kafka event flow with a single event broker or cluster. You create modeled event meshes to help you define and visualize event flows between publishing and subscribing applications within your event mesh.
Runtime Event Manager	Runtime Event Manager is the tool in Event Portal where you build modeled event meshes, collect runtime data discovered by Event Management Agents, and audit the runtime data against modeled event meshes.

This section includes the following tasks:

• Setting Up Runtime Data Discovery
• Setting Up Event Management Agent Connections
• Setting Up Event Management Agents
• Running Discovery Scans
• Viewing the Collected Data
• Viewing the Collection Logs
• Deleting Discovery Scan Data
• Updating an Event Management Agent
• Deleting an Event Management Agent

For information about auditing the collected data and importing objects into Designer, see Auditing and Importing Runtime Data.

Setting Up Runtime Data Discovery

Before you can discover runtime data from an event broker, you need to complete several tasks. These tasks include:

1. Create a modeled event mesh in Runtime Event Manager. For more information, see Creating a Modeled Event Mesh.

2. Add a model event broker that represents the runtime event broker in your modeled event mesh. For Solace modeled event meshes, you can add more than one event broker. For more information, see Adding an Event Broker to a Modeled Event Mesh.

3. Create an Event Management Agent connection for one or more event brokers. See Setting Up Event Management Agent Connections.

4. Set up an Event Management Agent on a computer that has access to your runtime event brokers. If you want to upload discovery scan data to Event Portal manually, you also set up a second Event Management Agent in a location that allows it to connect to Event Portal. See Setting Up Event Management Agents.

After you've completed the setup, you can begin discovering and auditing runtime data.

Setting Up Event Management Agent Connections

Before you set up Event Management Agents to connect Event Portal with one or more runtime event brokers, you first set up an Event Management Agent connection. You can also add Confluent Schema Registry details to Kafka event broker connections if you want to import Confluent subjects as schemas.

Each Event Management Agent connection allows you to generate a file with all of the event broker connection details required by the Event Management Agent. The agent uses the connection details to scan the specified runtime event brokers and, if configured, a Confluent Schema Registry. The Event Management Agent can scan all event brokers supported by Solace, event brokers for Kafka version 2.5 and later, and Confluent Schema Registry version 6.2 and later.

To set up an Event Management Agent connection, perform these steps:

1. Log in to the PubSub+ Cloud Console if you have not done so yet. The URL to access the Cloud Console differs based on your authentication scheme. For more information, see Logging into the PubSub+ Cloud Console.
2. In the Cloud Console, select Runtime Event Manager on the navigation bar.
3. Select the Event Management Agents tab.
4. Click Set Up Event Management Agent
5. Enter a Name for the Event Management Agent.

6. Select a Discovery Scan Mode.

   • Scan from Event Portal uses one Event Management Agent that is connected to both Event Portal and to one or more runtime event brokers. This mode allows you to run discovery scans from Event Portal and automatically upload the scanned data.

   • Upload Scan File uses two Event Management Agents; one is connected to Event Portal and the other is connected to one or more runtime event brokers. The agent connected to your event brokers runs the discovery scan. You upload the scan file to Event Portal manually by copying it to the connected agent.

7. Click Manage Event Brokers to select one or more event brokers.

   The list shows all model event brokers and all Solace event broker services that are not already associated with an Event Management Agent. You can filter the list by name and by event broker type.

8. Click Add next to the event brokers you want to create a connection for. If the event broker has not already been added to a modeled event mesh, see Adding an Event Broker to a Modeled Event Mesh.
9. If you need to create a new model event broker, perform these steps:
   1. Click Add New Connection.
   2. Select Solace or Kafka as the event broker Type.
   3. Enter a Name for the model event broker.
   4. Add the connection details for the event broker. For more information, see the instructions to add an event broker to a modeled event mesh.
   5. Click Connect Event Broker.

10. Click Save and Set Up Agent.

    Runtime Event Manager begins walking you through setting up the Event Management Agent.

Setting Up Event Management Agents

You set up an Event Management Agent on a computer that can connect to your runtime event brokers. If the computer doesn’t have Internet access to also connect to Event Portal, you need to set up a second agent on a computer with Internet access.

After you set up an Event Management Agent connection, Runtime Event Manager walks you through the Event Management Agent set-up process for the discovery scan mode you chose:

• If you selected Scan from Event Portal as the Discovery Scan Mode mode, Runtime Event Manager guides you to set up one Event Management Agent that is connected to both Event Portal and one or more runtime event brokers. For more information, see Setting Up an Event Management Agent to Run Scans from Event Portal.

• If you selected Upload Scan File, Runtime Event Manager guides you to set up two Event Management Agents, one connected to runtime event brokers and the other connected to Event Portal. The agent connected to your event brokers runs the discovery scan. You upload the scan file to Event Portal manually by copying it to the connected agent. For more information, see Setting Up Event Management Agents to Upload Scanned Files.

If you need to update an Event Management Agent to either upgrade the agent or update the event broker connection information, see Updating an Event Management Agent.

Running Discovery Scans

After you've set up an Event Management Agent, it can run discovery scans on all of the runtime event brokers that it has a connection to. The Event Management Agent collects the following data:

• Queue and subscription configuration data for Solace event brokers

• Topic and consumer group configuration data for Kafka clusters

• Subject data if the Event Management Agent is configured to connect to a Confluent Schema Registry

You can view the raw data and collection logs for each discovery scan and view the audit results comparing the data from a runtime event broker with the corresponding model event broker. You can also use the Event Portal v2 REST API to get configuration and configuration type details from the collected data.

You can run a new discovery scan to collect the most recent data at any time. New scans are not incremental. Each scan collects the full set of data from the event broker.

If your Event Management Agent uses Upload Scan File mode, see Running and Uploading a Discovery Scan for instructions to run discovery scans and upload the data to Event Portal.

To run a discovery scan from Runtime Event Manager using Scan from Event Portal mode, perform these steps:

1. On the Runtime Event Manager home page, select the Modeled Event Meshes tab.
2. Click the name of the modeled event mesh that you want to collect runtime data for.
3. Select the Event Broker Connections tab.
4. Select the event broker that you want to collect runtime data for.

5. Click Run Discovery Scan.

   For Solace event brokers, the scan starts.

6. For Kafka event brokers perform the following actions:
   1. To scan both the event broker and the schema registry, select Scan All to start the scan.
   2. To scan only one of the event broker or schema registry, select Custom Scan.
   3. Select Broker or Confluent Schema Registry.
   4. Click Collect Data.

Viewing the Collected Data

You can view the raw data collected data from a discovery scan in Runtime Event Manager.

To view the raw data:

1. On the Runtime Event Manager home page, select the Modeled Event Meshes tab.
2. Click the name of the modeled event mesh that you want to view the runtime data for.
3. Select the Event Broker Connections tab.
4. Click More Actionsfor the discovery scan that you want to view the data for and select View Collected Data.

Viewing the Collection Logs

You can view the data collection logs in Runtime Event Manager to help troubleshoot connection issues.

To view the collection logs:

1. On the Runtime Event Manager home page, select the Modeled Event Meshes tab.
2. Click the name of the modeled event mesh that you want to view the collection logs for.
3. Select the Event Broker Connections tab.
4. Click More Actionsfor the discovery scan that you want to view the logs for and select View Logs.

Deleting Discovery Scan Data

You can delete older discovery scan data from Runtime Event Manager when it is no longer useful.

To delete discovery scan data:

1. On the Runtime Event Manager home page, select the Modeled Event Meshes tab.
2. Click the name of the modeled event mesh that you want delete discovery scan data for.
3. Select the Event Broker Connections tab.
4. Click More Actionsfor the discovery scan that you no longer need the data for and select Delete.
5. Click Delete.

Updating an Event Management Agent

You may need to update an Event Management Agent to add event brokers or schema registries, update connection details, or upgrade the Event Management Agent itself. If an Event Management Agent is not the latest version. Runtime Event Manager displays an upgrade notification in the Event Management Agents list:

Event Management Agent version 1.2.0 and later displays the version number in the information panel.

To update Event Management Agent connection details, perform these steps:

1. On the Runtime Event Manager home page, select the Event Management Agents tab.
2. Click More Actionsfor the Event Management Agent that you want to update.
3. Select Edit Connection Details.
4. Update the Event Management Agent settings as required. You can update everything except the Discovery Scan Mode.
5. Repeat the steps for Setting Up Event Management Agent Connections and Setting Up Event Management Agents
6. Repeat the installation

To upgrade an Event Management Agent that uses Scan From Event Portal mode without updating the connection details, you need only to pull a new Event Management Agent image from Docker and restart it:

1. Pull the Event Management Agent image from Docker Hub using the following command:

   docker pull solace/event-management-agent

2. Start the agent in your OS terminal using the following command:
   • Replace /path/to/file/AcmeRetail.yml with the path to your connection file.

   docker run -d -p 8180:8180 -v /path/to/file/AcmeRetail.yml:/config/ema.yml 
   --name event-management-agent solace/event-management-agent:latest

If an Event Management Agent uses Upload Scan File mode, see Setting Up the Event Management Agents to upgrade the Event Management Agent

Deleting an Event Management Agent

If you no longer need an Event Management Agent, you can delete the Event Management Agent Connection in Event Portal and then uninstall the Event Management Agent.

To delete an Event Management Agent Connection, perform these steps:

1. On the Runtime Event Manager home page, select the Event Management Agents tab.
2. Click More Actionsfor the Event Management Agent that you no longer need.
3. Select Delete from Event Portal.
4. Click Delete Connection.