Skip to main content

Xference AI Model Configuration in ConvyAI

Overview​

Xference brings LLM inference to dedicated Italian infrastructure, letting you adopt generative capabilities without conversation data ever leaving Italian territory. It's built for contexts where data localization is a requirement, not an option.

Prerequisites​

Create an Xference AI Model​

Navigate to Generative AI → AI Models and click New Model → Xference.

Configuration options​

FieldDescription
AI Model nameA unique name to identify this AI Model.
Check as defaultMarks this AI Model as the default one for the tenant.
Fast recognitionEnables fast recognition mode for this model.
Xference Api KeyThe API key for your Xference instance. Masked once saved.
Xference EndpointThe URL of your Xference instance.
Intent thresholdMinimum confidence score for an intent to be considered valid.

Advanced parameters​

ParameterDefaultDescription
Prompt versionv3System prompt version applied when using the model in Knowledge Base nodes.
Completion model-The completion model to invoke. Enter one of the models currently exposed by Xference (Qwen 3.6, Gemma 4 3B, Gemma 4 12B) or any other model name, as a free-text field.
Embedding modelBAAI/bge-m3The embedding model exposed by Xference, used during Knowledge Base indexing and retrieval.
Thinking Budget-The token budget for the reasoning of the model, as a whole number. Leave empty to use the model default, enter 0 to turn reasoning off, or enter a value to set the budget. The value applies to both Knowledge Base and AI Agent nodes, so you configure it once.
Max token completion-Maximum number of tokens in the generated response.
Temperature-Controls response creativity. Lower values (0.0–0.3) produce more deterministic outputs; higher values (0.7–1.0) produce more varied responses.
Top P-Nucleus sampling parameter. Limits token selection to the top probability mass.
Frequency penalty-Reduces repetition of tokens that have already appeared in the output.
Presence penalty-Encourages the model to introduce new topics by penalizing tokens that have already appeared.
Number of extracted chunks3How many Knowledge Base chunks are retrieved and passed to the model as context during RAG.

Optional parameters start empty on new AI Models. An empty field means the parameter is not sent and the model default applies; any value you enter, including 0, is sent as is. See Optional parameters.

Using an Xference AI Model​

Once configured, an Xference AI Model can be used wherever a generative AI Model is selected:

  • In the AI Agent node of the FlowDesigner, and in the AI Agent node of the Agentic Designer - including MCP Server tools, JavaScript functions, and image analysis.
  • In the Knowledge Base node, for intent classification and RAG-grounded responses.

Intents, entities, and testing​

Configuring intents and entities and testing the model are features shared across all AI Model providers. See Intents, Entities & Testing for full documentation.

Notes and limitations​

  • Thinking Budget is a token budget, not a level like the reasoning effort of other providers. Whether and how a model uses it depends on the model: use the test console to verify the behavior of your model before deploying.
  • The reasoning of the model never appears in the response returned to the end user.
  • ConvyAI does not report the reasoning tokens actually consumed, and whether the budget is respected depends on the provider. If the budget is too low for the response, the model can return an incomplete or empty answer: the test console shows the messages returned by the provider.
  • AI Models already configured keep their saved values. Review Max token completion, which is no longer prefilled with 1000 on new AI Models.
  • An index built with the BAAI/bge-m3 embedding model is not compatible with indexes built with a different embedding model. Adopting Xference on an existing Knowledge Base requires creating a new index and fully reindexing the content. See Indexing.