Skip to main content

Generative AI Statistics - Review AI Interactions with Full Traceability in ConvyAI

Overview​

The Generative AI section lets you search for conversations that involved the Generative AI service and inspect every detail of each interaction - from the model invoked to the knowledge base chunks retrieved, to the full AI Agent response chain.

Navigate to Monitor → Generative AI.

Filter interactions by:

  • Bot name
  • Dialog ID
  • User ID
  • AI Model
  • Feedback (user rating of the AI response)

Results list​

The results list shows each interaction with:

  • Date and time of the interaction.
  • Dialog ID.
  • The user's input.
  • The AI-generated output.
  • A button to open the full interaction details.

Results can be exported as CSV or XLSX. Use the Group by Dialog ID button to group all interactions from the same conversation together.

Interaction details​

Click the details button to open the full view for a conversation. The left panel lists all interactions within that dialog, grouped by flow node. Click any interaction to expand its data across four sections.

Interaction​

FieldDescription
Dialog IDWith a direct link to the Dialog Details screen.
Request / response date and timeTimestamps for the AI call.
Global interaction counterThe count of generative AI interactions at the full conversation level.
Node interaction counterThe count of interactions at the individual node level.
Bot detailsBot ID, name, and associated flow.
Node IDThe flow node that triggered this interaction.
User questionThe input sent by the user.
AI responseThe generated answer.
OutcomeWhether the request to the generative model succeeded or failed.
Response timeTime taken to obtain the AI response.
SentimentThe sentiment extracted from the user input.
User feedbackThe rating or feedback provided by the user.

AI Model​

Information about the model used:

FieldDescription
Model typeThe provider (OpenAI, Bedrock, Gemini, etc.).
Model nameThe specific model version invoked.
Invocation parametersTemperature, top-p, and other generation parameters used.

Insight​

FieldDescription
Extracted intentThe intent recognized in the user's message, with its relevance score.
Exit reasonWhy the generative node was exited - either a recognized intent or the maximum interaction count reached.
Input languageThe language detected in the user's input.
Extracted entitiesAny entities pulled from the user's message.

Documents​

Shows the knowledge base sources used to generate the response:

FieldDescription
Document nameThe source document from which the response was generated.
Real-time analysisWhether the real-time document analysis feature was enabled.
Knowledge base indexThe index name queried.
Text chunksThe individual chunks retrieved from the knowledge base and passed as context. For each chunk, the vector distance from the user's input embedding is shown.

Report AI (AI Agent tracing)​

The Report AI section provides a full trace of each interaction generated by an AI Agent node - including the complete response chain showing which agents and tools were involved in processing.

Features:

  • Chronological visualization of all interactions in the chain.
  • Tracking of AI requests and responses at each step.
  • Tool details - inputs and outputs for each tool called during processing.
  • Response chain visualization - highlights each agent and resource engaged in the final output.
  • Copy functionality to extract content for further analysis.
tip

The Report AI section is the primary resource for debugging, monitoring, and optimizing AI Agent-driven interactions. It provides full transparency into the reasoning flow and the tools invoked at each step.