Skip to main content

Knowledge Base - RAG Architecture and Indexing Pipeline in ConvyAI

What is the Knowledge Base​

The Knowledge Base is the content layer that contextualizes Generative AI responses in ConvyAI. Instead of relying on generic model training data, you index your own documents and website pages. The platform then uses that indexed content to ground every LLM response in verified, brand-aligned information.

This approach - known as Retrieval Augmented Generation (RAG) - combines the language fluency of large language models with the precision of a curated content source.

note

Access to Knowledge Base features requires activation. Contact the ConvyAI Sales team to enable this functionality for your account.

How it works​

Indexing phase​

Before a bot can draw from the Knowledge Base, content must be indexed:

  1. Content extraction - Text is extracted from documents (PDF, DOC) stored in an AWS S3 bucket, or from pages crawled from a website.
  2. Chunking - The extracted text is split into smaller chunks. Chunk size is configurable.
  3. Embedding generation - Each chunk is converted into a vector embedding - a numerical representation that captures the semantic meaning of the text.
  4. Vector storage - Embeddings are stored in a vector database (Redis) alongside the original text chunks.

The diagram below shows how content flows from its source to the vector database during indexing:

Retrieval phase​

When a customer sends a message to the bot:

  1. The user's input is converted into a vector embedding using the same embedding model used during indexing.
  2. A similarity search compares the input embedding against all stored vectors.
  3. The most semantically relevant chunks are retrieved.
  4. The retrieved chunks are passed to the LLM as context, alongside the user's message and any configured instructions.
  5. The LLM generates a response grounded in the retrieved content.

The diagram below shows how a user message travels through the retrieval pipeline to produce a grounded response:

Index structure​

ConvyAI supports two indexing strategies:

StrategyDescriptionWhen to use
Single indexAll content is indexed under one name. Bots draw from a single unified knowledge domain.Simple use cases with a homogeneous knowledge base.
Multiple indexesContent is organized under different index names (e.g., by product line, topic, or language). Each bot node can specify which index to query.When you need to scope responses to a specific knowledge domain.
  • Settings - Configure the API credentials and vector database connection.
  • Indexing - Create and run indexing batches.
  • Generative AI Bot - Use the Knowledge Base index in the Flow Designer.