Skip to main content

Watchbot - Non-Regression Testing for Bot Flows in ConvyAI

Overview​

Watchbot is ConvyAI's automated testing tool for conversation flows. It lets you save conversation scripts and re-execute them periodically to verify that flow changes have not broken expected behavior. A script that fails means the flow has changed in a way that no longer produces the expected output.

Navigate to Tools → Watchbot.

Create a test script​

  1. Execute a flow in the debug panel.
  2. In the debug panel, locate the Watchbot Scripts section.
  3. Enter a name for the script and save it.

The script records the exact sequence of inputs and expected text outputs from that conversation.

Manage scripts​

The Watchbot page lists all saved scripts.

Click on a script panel to expand it and view:

  • The script content - the recorded conversation steps.
  • Past executions - the history of previous runs with pass/fail status.

On the right side of the panel you can:

  • View and edit script variables.
  • Delete the script.
  • Execute the script in real time.
  • Modify the script if it has failed.

Interpret results​

A script that fails is marked with a red thumbs-down icon. For single-type scripts, click the icon to open the details modal.

The details modal shows:

  • The full conversation executed during the test.
  • The point where the output diverged from the expected result.

From the details modal you can:

  • Restore the script to its previous state.
  • Modify the script to adapt it to the updated flow behavior.
note

A Watchbot script expects the exact same text output produced during the initial test recording. Any change in the flow's text output - even a minor wording update - will cause the script to fail.