Watchbot - Non-Regression Testing for Bot Flows in ConvyAI
Overview
Watchbot is ConvyAI's automated testing tool for conversation flows. It lets you save conversation scripts and re-execute them periodically to verify that flow changes have not broken expected behavior. A script that fails means the flow has changed in a way that no longer produces the expected output.
Navigate to Tools → Watchbot.
Create a test script
- Execute a flow in the debug panel.
- In the debug panel, locate the Watchbot Scripts section.
- Enter a name for the script and save it.
The script records the exact sequence of inputs and expected text outputs from that conversation.
Manage scripts
The Watchbot page lists all saved scripts.
Click on a script panel to expand it and view:
- The script content - the recorded conversation steps.
- Past executions - the history of previous runs with pass/fail status.
On the right side of the panel you can:
- View and edit script variables.
- Delete the script.
- Execute the script in real time.
- Modify the script if it has failed.
Interpret results
A script that fails is marked with a red thumbs-down icon. For single-type scripts, click the icon to open the details modal.
The details modal shows:
- The full conversation executed during the test.
- The point where the output diverged from the expected result.
From the details modal you can:
- Restore the script to its previous state.
- Modify the script to adapt it to the updated flow behavior.
A Watchbot script expects the exact same text output produced during the initial test recording. Any change in the flow's text output - even a minor wording update - will cause the script to fail.