Skip to main content

AI Agent - LLM Orchestration in ConvyAI

What is an AI Agent​

The AI Agent is a major evolution of ConvyAI's Generative AI capabilities. Where the Knowledge Base node is designed for conversational RAG - retrieving content and generating grounded answers - the AI Agent is built for process orchestration: it lets the LLM interact with external tools, analyze images, invoke custom functions, and make decisions within a workflow, all directly inside the FlowDesigner.

Through the AI Agent node, developers and advanced users can interact directly with LLM models, perform image analysis, and combine different tools and external services without building custom integrations or middleware. Everything happens inside the FlowDesigner in a controlled, auditable way - prompts, parameters, and model outputs are all configurable without leaving the development environment.

How it differs from the Knowledge Base node​

Both the Knowledge Base node and the AI Agent node use the same AI Models infrastructure. What changes is the purpose and the capabilities available.

Knowledge Base nodeAI Agent node
Primary useConversational RAG - grounded Q&A from indexed contentProcess orchestration - decisions, extraction, multi-tool workflows
Data sourcesIndexed Knowledge BaseDynamic inputs: text, images, external tools via MCP
External integrationsNoneMCP servers, callable JavaScript functions
Image analysisNot supportedSupported
Conversation memoryOptional, session-scopedConfigurable via node settings

When to use the AI Agent​

The AI Agent is the right choice when the model needs to do more than generate a conversational reply:

  • Ticket and request analysis - automatically classify and route incoming requests based on their content.
  • Structured data extraction - extract structured fields from free-text input, documents, or images.
  • Multi-tool workflows - combine LLM reasoning with external services connected via MCP.
  • Complex decision-making - use model output to drive flow branching based on multi-format inputs.

Architecture​

The AI Agent node acts as an orchestration hub. The LLM is not just a response generator - it decides which tools to call, in which order, and how to use their outputs to produce a final result.

In this section​

  • AI Agent Node - Configure the node fields, prompt, tools, and interaction settings.
  • Image Analysis - Attach and analyze images within the agent prompt.
  • MCP Servers - Connect external tools and services via Model Context Protocol.
  • Callable Functions - Define custom JavaScript functions the AI can invoke during execution.