Designing and Managing Events with PubSub+ Event Portal

This series of tutorials will help you get familiar with PubSub+ Event Portal toolset and its Web user interface (UI). As we guide you through the tools and components in PubSub+ Event Portal(or simply Event Portal), we will dive into some of the activities you will perform when designing your event-driven architecture (EDA).

For a quick demo, check out the PubSub+ Event Portal Get Started Demo video.

Before you begin

Adding / Editing User Roles and Permissions

Account administrators can provide Event Portal access to a new user or edit the existing user's roles and permissions. Note that the Administrator, Event Portal Editor, and Event Portal Viewer user roles will all count towards the Event Portal's maximum user limit.

For more information and tutorials, refer to Editing Roles and Permissions.

Managing Application Domains

The Application Domain enables you to split the enterprise into sub-components. These sub-components could represent different lines of business, development teams, or microservice applications. The purpose is to enclose the events and schemas that you may or may not wish to share with other application domains.

The following tutorials outline the activities you will perform when creating, importing or exporting application domains.

Creating an Application Domain

To create an application domain, do the following:

  1. Navigate to the Designer tab and click the Create icon on the top right, or right click on the background.

  2. In the dialog that appears, provide the Name, Topic Domain and Description for the application domain.

    Optional:Under the Topic Domain field, the checkbox ensures all topics within that domain are unique. The checkbox is enabled by default. Leaving this feature enabled avoids the creation of multiple events having the same topic; therefore, preventing potential conflicts between event types in an environment.

    The Topic Domain field is important to understand and consider. For all the events defined within the application domain, the Topic Domain is added at the beginning by default, which is helpful in the creation of runtime Access Control Lists (ACLs). For further design considerations on Topic Domain, refer to the Topic Architecture Best Practices guide.

  3. Click Save. Your newly created application domain will be saved and visible on the screen.

Importing an Application Domain

To import an Application Domain from your computer, do the following:

  1. Navigate to the Designer tab and click the Import icon on the top right.

  2. Click the Import button and select the Application Domain JSON file you want to import from your computer.

    Once imported, the application domain will be visible in the event designer.

Exporting an Application Domain

The steps below outline the different ways you can export an application domain:

  1. Navigate to the Designer tab.
  2. Once you are in Designer, there are few ways you can export an application domain.

    Method 1:

    1. Right-click on the Application Domain you want to export.
    2. Select Export from the menu options and follow the prompt.

    Method 2:

    1. Select the application domain you want to export.
    2. On the domain panel that will appear on the top right, click Go to this domain.
    3. Once you are in the application domain, do one of the following:

      • Right click on the gray background
      • On the panel located on the top right, click the ellipsis (three dots)

    4. Select Export from the menu options and follow the prompt.

The Application Domain as a JSON file will be downloaded to your computer. You can reuse it to import into a different instance of the Event Portal or keep it as a backup.

Managing Applications, Events and Schemas

Event-driven architectures are made up of three fundamental and foundational elements or objects—Applications, Events, and Schemas—that are fully managed by the Event Portal. The tutorials below will show you how to create and use these entities. For overview information, take a look at PubSub+ Event Portal Overview.

In the tutorials below, we will walk you through the basic steps to create an application, event and schema. In doing so, we will also show you have to configure some of the Kafka-native attributes that will help in the design-time architecting of Kafka's event-driven architectures (EDAs), and in discovering EDAs directly from Kafka clusters.

Creating an Application

Applications represent the program, process, microservice, IoT device, integration component or other runnable consumers, producers or processors that interact with the event broker.

To create an application, do the following:

  1. Select the Application Domain where you want to create the application.
  2. On the panel that will appear on the top right, click View Domain Details or double-click on the application domain you want to drill into.

  3. Once you are inside the application domain, click the Create + icon located on the top right or right-click on the background, and select Create Application.
  4. In the pop-up dialog that appears, complete the details, as shown in the example below.

    • Name: The name of the application.
    • Description: Provide information about the Application. It is important to thoroughly explain the application, so that its proper usage and business implications are clearly understood.
    • Application Class: (optional) Specify the application type. You can select the Kafka Connector, Kafka Application or leave it Unspecified. See Step 5 for more information.
    • Owners: Add or remove other members of your organization as application owners.
    • Tags: Add or remove application tags. Learn more about tags and its usage in the overview section (Tags)
  5. In the Application Class field, if you have specified Kafka Connector or Kafka Application, you will need to provide additional information.
    • Kafka Connector: A connector is used in Kafka for connecting Kafka brokers with external systems to stream data into or out of Apache Kafka. In the Event Portal, a Kafka Connector is an application class you select to configure associated published and/or subscribed events and a set of Kafka-native attributes as described below, as shown below.

    • Kafka Application: You can select a Kafka Application as an application type to add consumer groups that you can associate with event subscriptions. See Configuring a Consumer Group to learn more.

  6. Click Save.

Configuring a Consumer Group

Event Portal supports the concept of Kafka's consumer groups. A consumer group is used by Kafka to group consumers into a logical subscriber for a topic. In Event Portal, you can model consumer groups in Designer. This enables the Event Portal's runtime discovery to associate a discovered consumer group to an existing application.

The tutorials below show you how to add a consumer group and subscribe to an event topic.

  1. You can use an existing application or refer to Creating an Application to create a new one.
  2. In the Application Class field, click the dropdown menu and select Kafka Application.

  3. Click Add Consumer Group, and in the field that is displayed in the Events Subscribed box, write the name of the consumer group.
  4. Click the Manage button to subscribe to an event that's already published in the Application Domain or shared from a different Application Domain. A list of available events will be displayed.

  5. Locate the event you want the consumer group to subscribe to and drag-and-drop the event on to the newly created consumer group.
  6. Click Save. The application will now subscribe to the event, which you can see on the application domain's Graph View.

Creating a Schema

You can create or import existing payload schemas in either JSON, AVRO, or XML schema. If your payload does not adhere to the above formats, you can choose Binary or Text. Make sure to provide a description to help developers with serialization/deserialization.

To create a new schema, complete the following steps:

  1. Select the Application Domain where you want to create the schema.
  2. On the domain panel that will appear on the top right, click View Domain Details or double-click on the application domain you want to drill into.

  3. Once you are in the application domain, click Create + icon located on the top right or right-click on the background and select Create Schema.

  4. In the dialog that appears, complete the details, as shown in the example below.

    • Name: The name of the schema
    • Content Type: JSON, AVRO, XML, Binary or Text.
    • Shared: Allows the schema to be referenced by events in other application domains. It is best practice to enable this feature only for schemas that follow a more rigorous governance process as its now an exposed interface. The shared options is disabled by default.
    • Description: Information about the schema.
    • Versions: Create and manage multiple versions of a schema. See, Creating Multiple Versions of a Schema for more information.
    • Owners: Add or remove other members of your organization as schema owners.
    • Tags: Add or remove schema tags. Learn more about tags and its usage in the overview section (Tags)

    • Content: Write a new schema or import it from a file. If you are using JSON Schema, the right screen dynamically generates a sample that you can use as a guide to enhance the readability of your schema design. Additionally, we support semantic error checking of the schema when developing with JSON Schema.

      For this example, we have used the following JSON schema.

  5. Click Save to complete the schema creation.

Creating Multiple Versions of a Schema

You can create and store multiple versions of the same schema, which can be used in your EDA designs that require multiple versions of a particular schema.

To create multiple versions of a schema, do the following:

  1. Edit an existing schema or refer to Creating a Schemato create a new one.
  2. Once you complete the top fields, navigate to Versions, and click the + icon.

  3. Write the version number. The version must be an integer or follow semantic versioning (e.g., 1, 1.0).
  4. Write your schema or import from file.
  5. Click Save.

You can save revisions of each schema version, see changes in the revisions, and revert to an older revision. To learn more about revision history, see Managing Object Revisions.

Creating an Event

The event is where you define metadata that describes and categorizes the event. From a modelling perspective, events reference payload schemas, and events are referenced by application, thus forming the glue that binds applications together.

To create an event, complete the following steps:

  1. Select the Application Domain where you want to create the event.
  2. On the domain panel that will appear on the top right, click View Domain Details or double-click on the application domain you want to drill into.
  3. Inside the application domain, click the Event icon on the top right or right-click on the background and select Create Event.
  4. In that dialog that appears, complete the details, as shown in the example below.

    • Name: The name of the event
    • Version: Version control (coming soon)
    • Shared: Allows the event to be referenced by applications in other application domains. It is best practice to enable this feature only for events that follow a more rigorous governance process as its now an exposed interface. The shared options is disabled by default.
    • Description: Information about the event. It is important to thoroughly explain the change in state (a business moment) and stimuli that kicks off the event so that consumers understand its proper usage and business implications.
    • Topic Scheme: Specify the event's Topic Scheme as Kafka, Solace (AMQP, REST, SMF), or Other.
    • Topic: The topic field provides the categorization of events in runtime. By default, the Application Domain’s Root Topic will be provided in the field so it can serve as a consistent prefix for naming.

      For now, the Topic field for a Solace Topic Scheme should only contain static parameters using the “/” as the level delimiter. In the future release, the Event Portal will support Dynamic Parameters. To get more information about designing topics, refer to Topic Architecture Best Practices guide.

    • Payload Schema: Select the payload schema for the event.
    • Key Schema and Value Schema: (optional) In Kafka, when a message is published, there are two parts to it: the key and the value. Each of the key and the value can have a schema defined for them to allow the Kafka clients to know how to serialize/deserialize the message. In Event Portal, you can create and use both the key and value schemas to accurately model the schemas for Kafka events.
    • Owners: Add or remove other members of your organization as event owners.
    • Tags: Add or remove event tags. Learn more about tags and its usage in the overview section (Tags)
  5. Click Save to complete the event creation.

Adding Events to your Application

When modelling applications in Event Portal, you need to add events to your application. The following tutorial will walk you through the steps to add an event to the Events Subscribed or Events Published list for your application.

In addition, you can also search for events using keywords to filter your results. For information on Event Portal's search capabilities, refer to Searching for Entities.

To add events to the Events Published or Events Subscribed list for your application, do the following:

  1. Navigate to the Designer tab and select an application domain.
  2. Inside the application domain, select an application. The Events panel will appear on the top right.
  3. Click the Manage link on the Events panel. The Manage Events dialog will appear with the list of available events.
  4. Once you've identified the event you want to use, select the Pub or Sub icon. You can also add events to both the Events Published or Events Subscribed list simultaneously. Note that the only events visible to select are either already attributed to the Application Domain you are in or Events shared by other application domains.

    : Search events using keywords to filter your results and make it easier to find what you are looking for. For more information, refer to Searching for Entities.

  5. Click the Select Events button. The selected event will be added to the Events Published, or the Event Subscribed list.
  6. Click Save.

Searching for Entities

As enterprises grow, the number of schemas, events and applications will increase to the point where browsing a list is no longer reasonable. Event Portal provides a rich search capability that enables you to always find what you are looking for with Catalog. In this quick tutorial, we provide an overview of Catalog's search capabilities, and a few tips to help you find things more easily.

To search the Event Portal for an event, application or schema, do the following:

  1. Navigate to the Catalog tab.
  2. At the top of the screen, choose from Schemas, Events or Applications on which you would like to perform the search.

  3. Click on the search field and type your search term. The Event Portal looks for content in all objects of that type; within the metadata or schema (if searching schemas).
  4. Once you have found an application, event, or a schema of interest, perform any of the following tasks:
    • Click on an application, event, or a schema to view its corresponding relationships.
    • Click the checkbox to see the revision history, add events to an application, add a schema to an event, or export an application's AsyncAPI specification document.

The ability to view relationships is an important feature. For example, a data scientist may know the name of a JSON Schema attribute and could search schemas for that keyword. From there, he can locate the schema of interest and view the events which reference that schema. Furthermore, the user can now decide to consume one of these events, depending on the business context of the event and the problem they are trying to solve.

Generating an AsyncAPI

You can generate an AsyncAPI document for each application in a relationship graph. To generate an AsyncAPI document for an application, perform the following steps:

  1. Navigate to Designer and open the application domain in which your application resides.

  2. Select the application for which you want to generate the AsyncAPI document.
  3. Right-click on the application, or on the panel located on the top right, click the ellipsis (three dots).
  4. From the menu options, select AsyncAPI.

  5. Choose your file format and click the Continue button. The Async API file will be downloaded to your computer.

With the downloaded AsyncAPI file, you can now use it was an input to a code generator to generate your event-driven application using the language and framework of your choice. You can learn more on their website at https://www.asyncapi.com.

Managing Object Revisions

The Event Portal tracks the history of changes made to objects defined in the Event Portal — Schema, Event and Application — by creating a new revision of the object each time it's modified. You can view the changes between different revisions and roll back to a previous revision if you need to.

Viewing the Revision History

To view or edit the revisions of an application, event, or a schema, do the following:

  1. Navigate to the Catalog tab or the Designer tab in the Event Portal section.
  2. At the top of the screen, click on Schemas, Events or Applications.
  3. Select the desired event, schema, or application from the list.

  4. Click on the revisions icon at the top right of the screen to view the list of revisions for that object.

Viewing an Older Revision

You can view each revision to see the name of the person who made changes in that Rev. You can also see their change comments if they left any. When you select a previous revision of the object, you'll see a header, like the image below, at the top of the page. You can navigate and view changes for each revision.

The functions displayed on the revisions menu, similar to the above image, are described in the following table.

Function

Description

View Current Revision

View the latest revision of the object.

Compare with Previous

Coming Soon

Compare with Current

Coming Soon

Restore this Revision

Roll back the content of the object to the previous revision that you are viewing.

View Event Revisions

Return to the list of event revisions.

View Schema Revisions Return to the list of schema revisions.
View Application Revisions Return to the list of application revisions.

Previous and Next

View the previous or next revision of the object.

Restoring a Previous Revision

To restore an event, application, or a schema to a previous revision, do the following:

  1. Select the event, application, or schema that you want to restore to a previous version
  2. Click View Revisions on the top right.
  3. Select the version you want to restore and click the Restore or the Restore Revision icon.

Revision history is retained in Event Portal up to a max of 100 revisions per object before the oldest revisions is deleted. When you restore an older revision, a copy of the previous revision is automatically created. For example, if you restore Rev 39, a copy of Rev 39 is created, and that copy will become the new (current) revision.

Deleting an Object

You can delete a schema, event, or an application or restore deleted objects if required. Note that when a schema is deleted, all the revisions of that schema is also deleted.

To delete an object, do the following:

  1. Navigate to the application domain of your choice
  2. At the top of the screen, click on Schemas, Events or Application.
  3. Select the desired event, schema, or application from the list.
  4. Click the delete icon on the top right.

To restore a deleted object, do the following:

  1. Navigate to the application domain of your choice
  2. At the top of the screen, click on Schemas, Events or Application.
  3. Click the ellipsis (...) on the top right and follow the prompt.

  4. Select the object you want to restore and then click Restore.

Additional Resources