Schemas

A schema defines the structure and meaning of each data value in the payload of an event so that the publisher and all subscribers understand the structure and what each value means. Some events can use a primitive schema when, for example, all values in the event are boolean or integers. For events that require a complex schema that defines various data attributes and their relationships, you can create or import existing payload schema definitions in JSON, XSD, DTD, Avro, and Protobuf format.

Complex schemas can reference other schemas as long as both schemas have the same schema type and they are both shared or, if the schemas are not shared, they are both in the same application domain.

You can create new schemas in Designer and you can add schemas to Designer by importing Confluent subjects from your runtime data in Runtime Event Manager. For more information, see Importing Kafka Objects.

This section includes the following tasks:

You can also create custom attributes for schemas. For more information, see Using Custom Attributes

You can also move schemas to a different application domain. For more information, see Moving Objects Between Application Domains

Creating a Schema

To create a schema in Designer, perform these steps:

  1. Log in to the PubSub+ Cloud Console if you have not done so yet. The URL to access the Cloud Console differs based on your authentication scheme. For more information, see Logging In to the PubSub+ Cloud Console.
  2. On the navigation bar, select Designer.
  3. On the Application Domains page, click the application domain that you want to add a schema to.
  4. Click Components to switch to the list view.

  5. Select the Schemas tab then click Create Schema.

    Screenshot showing the settings described in the surrounding steps.

  6. Enter a Name for the schema.
  7. Select the Schema Type. The Schema Type can be JSON Schema, XSD, DTD, AVRO, or Protobuf.
  8. If you want events and schemas in other application domains to be able to reference this schema, select Allow Schema to be shared across application domains.
  9. (Optional) Add values for any schema custom attributes that have been set up in your organization.
  10. In the Version field, specify a semantic version value for the schema in the format major.minor.patch. The default is 0.1.0.
  11. (Optional) Enter a Version Name for the initial version of the schema .
  12. (Optional) Enter a Description for the schema. The Description field supports Markdown. Click toggle preview to preview the rendered text.
  13. (Optional) Add values for any schema version custom attributes that have been set up in your organization.
  14. In the Content field enter the schema definition or click Import From File to import an existing schema definition.
  15. (Optional) To reference another schema in Event Portal, perform these steps:
    1. Click Manage Referenced Schemas.
    2. Locate the schema you want to reference. The list shows schemas of the same type in the same application domain, and shared schemas from other application domains that you have access to. You can search by name and filter the list by application domain. If you selected Allow Schema to be shared across application domains, only shared schemas appear in the list. You can click the name of the schema to view more details.
    3. Click Add next to the schema version that you want to reference.
    4. If it's not already part of the schema definition, in the Content field, add a reference to the schema. For example, in a JSON schema, add "schema_parameter": { "$ref": "/path/referenced_schema" } for a schema parameter that references another schema.
  16. Click Save Schema.

Updating a Schema

When you update a schema, you can update an existing version or create a new version of the schema. Versions allow you to work on updates and test new versions in development and staging environments while latest stable version remains in the production environment. Each version also has a lifecycle state. You can only edit versions that are in Draft state. For more information, see Object Versions and Lifecycle States.

The version number uses semantic versioning in the format major.minor.patch. When you update an object, you have three options:

  • If the object version is in Draft state, you can update the version and retain the version number

  • If the object version is in Released, Deprecated, or Retired state, you can duplicate a version and increment the version number to create a new Draft version.

  • If the object version is in Released, Deprecated, or Retired state, you can create a new version of the object from scratch

If you want to update a version that is not in Draft state it is best practice to create a new version rather than returning the version to Draft state.

When you reference an object, for example, you specify the schema that an event uses or an event that an application publishes, you select the version of the object that you want to reference. If a new version of a referenced object is created, the reference does not update automatically. You can choose when to update references to the new version.

If you import a schema into Event Portal from a Confluent Schema Registry, the imported schema uses the registry version number as the major version. For example, version 1 in the registry is imported as version 1.0.0 in Designer. You can change the version number, version name, description, and custom attributes of imported schema versions in Designer, but you can't update the schema content or the list of referenced schemas. If necessary, you can create a new version of the schema object in Designer and update the new version. If you do add a version or change a version number in Designer, Solace recommends incrementing the minor or patch version number to avoid potential conflicts with future imported versions.

If allowed for objects in the application domain, the description of a schema version can be edited at any lifecycle state. This option allows you to add information to the description such as deprecation and retirement dates and notes about the object's functionality or runtime behavior.

To update a schema, perform these steps:

  1. On the Application Domains page,click the application domain that contains the scheme you want to update.
  2. Click Components to switch to the list view.
  3. Select the Schemas tab.
  4. In the list of schemas, click More Actions for the schema you that want to update and select Open Schema.
  5. Perform one of the following actions:
    • In the Versions list, click More Actions for the version that you want to update and select Edit. Any changes you make update that version of the schema. You can edit a schema version only if it is in Draft state.
    • In the Versions list, click More Actions for the version that you want to update and select Duplicate. All properties of the existing version are copied into a new version. By default, the new version number is incremented by 0.0.1. You can change the version number to any number not already in use.
    • Above the Versions list, click Addto create a new version of the schema without duplicating an existing version. By default, the new version number is the next unused major version. You can change the version number to any number not already in use.
  6. Update the schema version details as necessary.
  7. Click Save Version or Save & Close.

Changing the State of a Schema Version

When you create a new version of a schema, the version is in Draft state. When the version is ready, you can change the version state to Released. When you release a new version of a schema, you may also want to change older versions of the schema to Deprecated or Retired. You must have at least the Event Portal User role with Manager level access to the application domain to change the version state. For more information, see Object Versions and Lifecycle States

You must change a schema out of Draft state before you can change any events or other schemas that reference it out of Draft state. If you want to retire a schema version, you must first retire all event versions and other schema versions that reference it.

To change the state of a schema version, perform these steps:

  1. On the Application Domains page,click the application domain that contains the scheme you want to update.
  2. Click Components to switch to the list view.
  3. Select the Schemas tab.
  4. In the list of schemas, click More Actions for the schema that you want to update and select Open Schema.
  5. Select the version that you want to change the state of, then click Manage Lifecycle.
  6. Select the new state.
  7. (Optional) If you set the State to Deprecated or Retired, set the End of Life date. If you set a planned or actual End of Life date, the date displays in the version details.
  8. Click Save.