OpenAI-Compatible with Qdrant (Beta)

The following table describes the parameters for configuring a RAG Agent that uses an OpenAI-compatible LLM and embedding model with a Qdrant vector database.

Field	Description
OpenAI-Compatible Embedding Model Provider
Endpoint URL	The API endpoint URL for the embedding model provider. For example: `https://api.openai.com/v1/embeddings`
Model ID	The specific version or name of the embedding model to use. For example: `openai/text-embedding-3-large`
Secret Access Key	The API Secret Access Key for the embedding model provider.
Organization ID	The OpenAI Organization ID.
Chunk Size	The size of text segments into which a document is divided by an embedding model for vector database storage. Default: 1000.
Embedding Model Context Length	The maximum number of tokens to embed at once by the embedding model. Default: 8191.
Number of Embedding Dimensions	The number of dimensions in the resulting output embeddings.
Qdrant Vector Database
Endpoint URL	The API endpoint URL for the Qdrant Vector Database. For example: `https://xyz-example.eu-central.aws.cloud.qdrant.io:6333`
Secret Access Key	The API Secret Access Key for the Qdrant provider.
Collection Name	The name of the Qdrant collection. A collection in Qdrant is a container that stores vectors where all vectors must have the same dimensions and have used the same model for their creation. Collections allow you to efficiently search for and retrieve vectors that are similar to a vector you specify in a query. For more information, see Collections in the Qdrant documentation. If the specified collection does not exist, the Agent creates a new empty basic collection with the specified name.
Vector Database Retrieval Limit	The maximum number of similar documents to retrieve from the vector database.
Select LLM Provider	Not editable. Pre-populates to OpenAI-Compatible.
OpenAI-Compatible LLM Connection Details
Endpoint URL	The API endpoint URL for the LLM provider. For example: `https://api.openai.com/v1`
Model ID	The specific version or name of the LLM to use. For example: `openai/gpt-4o`
Secret Access Key	The API Secret Access Key for the LLM provider.
User Prompt	A template for the user prompt with placeholder `{{text://input.payload}}` for user input and placeholder `{{text://user_data.retrieved_data}}` for injection of vector database query results. For example: `<context>{{text://user_data.retrieved_data}}</context><user-question>{{text://input.payload}}</user-question>`
System Prompt	A system prompt that provides instructions for the model's overall behavior.
Max Tokens	The `max_tokens` hyperparamter, which controls the maximum length of the LLM's response.
Temperature	The `temperature` hyperparameter, which controls the randomness of the LLM's output. Valid values are between 0.0 and 1.0.
Destination for AI Responses
Destination Type	Specifies whether the destination on the event broker service is a topic endpoint or queue.
Destination Name	The name of the topic or queue to publish AI response messages to. For the Beta release, you can publish only to queues.

Provide feedback