LLM Processor (Beta)

This processor Micro-Integration consumes events from an event broker service, forwards them to an LLM, and then publishes the augmented events back to the event broker service.

At the Micro-Integration level, you must configure the connection parameters for your event broker service and LLM. For more information, see Micro-Integration Parameters.

You must also define one Micro-Integration Flow that specifies the endpoints where data is consumed from and published to. For more information, see Micro-Integration Flow Parameters.

This Micro-Integration supports only custom header mapping. Payload mapping is not supported.

Micro-Integration Parameters

For detailed information about connecting to your event broker service, see Event Broker Service Connection Parameters.

The following table describes the parameters for configuring an LLM processor Micro-Integration.

Field Description
OpenAI-Compatible LLM Connection Details

Endpoint URL

The LLM Provider API endpoint URL. Example: https://api.openai.com/v1

Model ID

The specific version or name of the LLM to use. Example: openai/gpt-4o

Secret Access Key

The API Secret Access Key for the LLM provider.

User Prompt

A template for the user prompt to request a particular response from the model. Use the {{text://input.payload}} placeholder to include the message payload in the prompt.

For example:

  • You are a helpful AI assistant. Please summarize the information within the <content> tags: <content> {{text://input.payload}}</content>

  • You are a senior customer support specialist. Score the comments in the <content> tags on a scale of 1 (very negative) to 5 (very positive): <content> {{text://input.payload}}</content>

Max Tokens

The max_tokens hyperparameter, which controls the maximum length of the LLM's response.

Temperature

The temperature hyperparameter, which controls the randomness of the LLM's output. Valid values are between 0.0 and 1.0.

Micro-Integration Flow Parameters

You must configure the endpoint parameters for the Micro-Integration Flow.

Source Endpoint

Field Description
Endpoint Type

Select TOPIC_ENDPOINT or QUEUE to specify the type of endpoint to receive from.

Queue Name

The name of the queue to receive from. Endpoint Type must be QUEUE.
Topic Name

The name of the topic endpoint to consume from. Endpoint Type must be TOPIC_ENDPOINT.
Message Selector For the event broker service as the source, enables message filtering per endpoint.

Target Endpoint

Field Description
Destination Type

The destination type to send AI responses to. Select TOPIC or QUEUE.

Create the required queue on your event broker service if you haven't done so already. For more information, see Creating Event Broker Services.

Destination Name The name of the queue or topic to publish AI responses to.