Xference AI Model Configuration in ConvyAI
Overview
Xference brings LLM inference to dedicated Italian infrastructure, letting you adopt generative capabilities without conversation data ever leaving Italian territory. It's built for contexts where data localization is a requirement, not an option.
Prerequisites
- An Xference endpoint URL and API key.
- Credentials configured in Generative AI Settings.
Create an Xference AI Model
Navigate to Generative AI → AI Models and click New Model → Xference.
Configuration options
| Field | Description |
|---|---|
| AI Model name | A unique name to identify this AI Model. |
| Check as default | Marks this AI Model as the default one for the tenant. |
| Fast recognition | Enables fast recognition mode for this model. |
| Xference Api Key | The API key for your Xference instance. Masked once saved. |
| Xference Endpoint | The URL of your Xference instance. |
| Intent threshold | Minimum confidence score for an intent to be considered valid. |
Advanced parameters
| Parameter | Default | Description |
|---|---|---|
| Prompt version | v3 | System prompt version applied when using the model in Knowledge Base nodes. |
| Completion model | - | The completion model to invoke. Enter one of the models currently exposed by Xference (Qwen 3.6, Gemma 4 3B, Gemma 4 12B) or any other model name, as a free-text field. |
| Embedding model | BAAI/bge-m3 | The embedding model exposed by Xference, used during Knowledge Base indexing and retrieval. |
| Thinking Budget | - | The token budget for the reasoning of the model, as a whole number. Leave empty to use the model default, enter 0 to turn reasoning off, or enter a value to set the budget. The value applies to both Knowledge Base and AI Agent nodes, so you configure it once. |
| Max token completion | - | Maximum number of tokens in the generated response. |
| Temperature | - | Controls response creativity. Lower values (0.0–0.3) produce more deterministic outputs; higher values (0.7–1.0) produce more varied responses. |
| Top P | - | Nucleus sampling parameter. Limits token selection to the top probability mass. |
| Frequency penalty | - | Reduces repetition of tokens that have already appeared in the output. |
| Presence penalty | - | Encourages the model to introduce new topics by penalizing tokens that have already appeared. |
| Number of extracted chunks | 3 | How many Knowledge Base chunks are retrieved and passed to the model as context during RAG. |
Optional parameters start empty on new AI Models. An empty field means the parameter is not sent and the model default applies; any value you enter, including 0, is sent as is. See Optional parameters.
Using an Xference AI Model
Once configured, an Xference AI Model can be used wherever a generative AI Model is selected:
- In the AI Agent node of the FlowDesigner, and in the AI Agent node of the Agentic Designer - including MCP Server tools, JavaScript functions, and image analysis.
- In the Knowledge Base node, for intent classification and RAG-grounded responses.
Intents, entities, and testing
Configuring intents and entities and testing the model are features shared across all AI Model providers. See Intents, Entities & Testing for full documentation.
Notes and limitations
- Thinking Budget is a token budget, not a level like the reasoning effort of other providers. Whether and how a model uses it depends on the model: use the test console to verify the behavior of your model before deploying.
- The reasoning of the model never appears in the response returned to the end user.
- ConvyAI does not report the reasoning tokens actually consumed, and whether the budget is respected depends on the provider. If the budget is too low for the response, the model can return an incomplete or empty answer: the test console shows the messages returned by the provider.
- AI Models already configured keep their saved values. Review Max token completion, which is no longer prefilled with
1000on new AI Models. - An index built with the
BAAI/bge-m3embedding model is not compatible with indexes built with a different embedding model. Adopting Xference on an existing Knowledge Base requires creating a new index and fully reindexing the content. See Indexing.