Skip to main content

AI Models - Configure LLM Providers for Generative AI in ConvyAI

What are AI Models​

An AI Model in ConvyAI is a configured connection to a Large Language Model provider. It defines which LLM engine to use, the authentication credentials, and how the model should behave - including intent and entity extraction parameters.

AI Models are the shared LLM infrastructure used across ConvyAI's Generative AI capabilities:

  • In the Knowledge Base node - to analyze user input, classify intent, extract entities, and generate RAG-grounded responses.
  • In the AI Agent node - to orchestrate tool calls, process images, and produce outputs that drive flow logic.

Supported providers​

ProviderModelsNotes
OpenAIGPT-3.5 and aboveRequires OpenAI API key
Amazon BedrockAnthropic Claude (multiple versions)Requires AWS credentials
Google GeminiGemini 3.5 and laterRequires Gemini API key
Azure OpenAIAzure-deployed OpenAI modelsRequires Azure endpoint and key
XferenceQwen 3.6, Gemma 4 3B, Gemma 4 12BRequires Xference endpoint and key; inference runs on Italian datacenters
Bring your own ModelAny model with a REST APIHTTP REST integration

Optional parameters​

Optional parameters of an AI Model (such as Temperature, Top P, penalties, reasoning effort, extended thinking budget, Thinking Level and Thinking Budget) follow the same rule for every provider and model:

  • An empty field means the parameter is not sent, and the provider default applies.
  • Any value you enter is sent to the provider as is.

ConvyAI does not disable, hide or force any parameter based on the model you enter. New AI Models start with these fields empty, and an information icon next to the fields reminds you of the rule. If a model does not accept a parameter, the provider rejects the call: use the test console to find out before deploying.

Reasoning settings (reasoning effort on OpenAI and Azure OpenAI, extended thinking budget on Amazon Bedrock, Thinking Level on Google Gemini, Thinking Budget on Xference) are configured once on the AI Model and apply to Knowledge Base nodes and AI Agent nodes alike, in both FlowDesigner and Agentic Designer.

AI Models created before version 7.9.0 keep their saved values and continue to send them.

In this section​