A one-line change to a playbook can change how an agent handles thousands of conversations. Most teams still check it by chatting with the agent a few times and hoping. Simulations replace hope with a repeatable rehearsal.
What a simulation is
A simulation plays a scripted customer against your agent, start to finish, with real actions pointed at test data. It checks the outcome you care about: was the refund issued, was the right policy cited, did the agent hand over when it should.
Build the suite from real conversations
- Start with the twenty most common journeys, taken from real transcripts
- Add every conversation that went wrong in the last month
- Add the edge cases your team argues about
A suite of fifty scenarios covers more ground than a week of manual testing, and it runs in minutes.
Make it a gate, not a report
Run the suite on every change and block the release when a scenario fails. In Parley, simulations run from the CLI or on publish, and a failed scenario shows the trace of exactly where the conversation went another way.
Run the suite on every change and block the release when a scenario fails.
Pair it with split testing
Simulations tell you a change is safe. Split testing tells you it is better. Ship passing changes to a small share of real traffic first, compare resolution against the main version, then roll out.
Keep the suite alive
Every handoff that should not have happened is a new scenario. Teams that add one or two a week end up with a suite that knows their customers better than any single person on the team.



