Generative AI Statistics - Review AI Interactions with Full Traceability in ConvyAI
Overview
The Generative AI section lets you search for conversations that involved the Generative AI service and inspect every detail of each interaction - from the model invoked to the knowledge base chunks retrieved, to the full AI Agent response chain.
Navigate to Monitor → Generative AI.
Search
Filter interactions by:
- Bot name
- Dialog ID
- User ID
- AI Model
- Feedback (user rating of the AI response)
Results list
The results list shows each interaction with:
- Date and time of the interaction.
- Dialog ID.
- The user's input.
- The AI-generated output.
- A button to open the full interaction details.
Results can be exported as CSV or XLSX. Use the Group by Dialog ID button to group all interactions from the same conversation together.
Interaction details
Click the details button to open the full view for a conversation. The left panel lists all interactions within that dialog, grouped by flow node. Click any interaction to expand its data across four sections.
Interaction
| Field | Description |
|---|---|
| Dialog ID | With a direct link to the Dialog Details screen. |
| Request / response date and time | Timestamps for the AI call. |
| Global interaction counter | The count of generative AI interactions at the full conversation level. |
| Node interaction counter | The count of interactions at the individual node level. |
| Bot details | Bot ID, name, and associated flow. |
| Node ID | The flow node that triggered this interaction. |
| User question | The input sent by the user. |
| AI response | The generated answer. |
| Outcome | Whether the request to the generative model succeeded or failed. |
| Response time | Time taken to obtain the AI response. |
| Sentiment | The sentiment extracted from the user input. |
| User feedback | The rating or feedback provided by the user. |
AI Model
Information about the model used:
| Field | Description |
|---|---|
| Model type | The provider (OpenAI, Bedrock, Gemini, etc.). |
| Model name | The specific model version invoked. |
| Invocation parameters | Temperature, top-p, and other generation parameters used. |
Insight
| Field | Description |
|---|---|
| Extracted intent | The intent recognized in the user's message, with its relevance score. |
| Exit reason | Why the generative node was exited - either a recognized intent or the maximum interaction count reached. |
| Input language | The language detected in the user's input. |
| Extracted entities | Any entities pulled from the user's message. |
Documents
Shows the knowledge base sources used to generate the response:
| Field | Description |
|---|---|
| Document name | The source document from which the response was generated. |
| Real-time analysis | Whether the real-time document analysis feature was enabled. |
| Knowledge base index | The index name queried. |
| Text chunks | The individual chunks retrieved from the knowledge base and passed as context. For each chunk, the vector distance from the user's input embedding is shown. |
Report AI (AI Agent tracing)
The Report AI section provides a full trace of each interaction generated by an AI Agent node - including the complete response chain showing which agents and tools were involved in processing.
Features:
- Chronological visualization of all interactions in the chain.
- Tracking of AI requests and responses at each step.
- Tool details - inputs and outputs for each tool called during processing.
- Response chain visualization - highlights each agent and resource engaged in the final output.
- Copy functionality to extract content for further analysis.
The Report AI section is the primary resource for debugging, monitoring, and optimizing AI Agent-driven interactions. It provides full transparency into the reasoning flow and the tools invoked at each step.