Knowledge Base - RAG Architecture and Indexing Pipeline in ConvyAI
What is the Knowledge Base
The Knowledge Base is the content layer that contextualizes Generative AI responses in ConvyAI. Instead of relying on generic model training data, you index your own documents and website pages. The platform then uses that indexed content to ground every LLM response in verified, brand-aligned information.
This approach - known as Retrieval Augmented Generation (RAG) - combines the language fluency of large language models with the precision of a curated content source.
Access to Knowledge Base features requires activation. Contact the ConvyAI Sales team to enable this functionality for your account.
How it works
Indexing phase
Before a bot can draw from the Knowledge Base, content must be indexed:
- Content extraction - Text is extracted from documents (PDF, DOC) stored in an AWS S3 bucket, or from pages crawled from a website.
- Chunking - The extracted text is split into smaller chunks. Chunk size is configurable.
- Embedding generation - Each chunk is converted into a vector embedding - a numerical representation that captures the semantic meaning of the text.
- Vector storage - Embeddings are stored in a vector database (Redis) alongside the original text chunks.
The diagram below shows how content flows from its source to the vector database during indexing:
Retrieval phase
When a customer sends a message to the bot:
- The user's input is converted into a vector embedding using the same embedding model used during indexing.
- A similarity search compares the input embedding against all stored vectors.
- The most semantically relevant chunks are retrieved.
- The retrieved chunks are passed to the LLM as context, alongside the user's message and any configured instructions.
- The LLM generates a response grounded in the retrieved content.
The diagram below shows how a user message travels through the retrieval pipeline to produce a grounded response:
Index structure
ConvyAI supports two indexing strategies:
| Strategy | Description | When to use |
|---|---|---|
| Single index | All content is indexed under one name. Bots draw from a single unified knowledge domain. | Simple use cases with a homogeneous knowledge base. |
| Multiple indexes | Content is organized under different index names (e.g., by product line, topic, or language). Each bot node can specify which index to query. | When you need to scope responses to a specific knowledge domain. |
Related topics
- Settings - Configure the API credentials and vector database connection.
- Indexing - Create and run indexing batches.
- Generative AI Bot - Use the Knowledge Base index in the Flow Designer.