parley
DocsPlatform
Resources
JournalIdeas on running AI supportCustomer storiesHow teams use ParleyGuidesStep-by-step playbooksChangelogWhat shipped, by versionHelp centerAnswers and support
CustomersPricing
LoginGet started
DocsPlatformCustomersPricingJournalCustomer storiesGuidesChangelogHelp center
Get startedLogin

Take charge of your AI customer experience

Get startedTalk to sales

Platform

Platform overviewChannelsPlaybooksIntegrationsAnalyticsPricing

Resources

DocumentationGuidesCustomer storiesJournalChangelogHelp center

Company

AboutCareersContact salesContact supportTerms of servicePrivacy

Security

Trust centerSOC 2ISO 27001GDPRHIPAAData processing
parley

© Parley. All rights reserved.

Journal · Engineering

Rehearse before you ship: using simulations as a release gate

Software teams don't ship without tests. Agent changes deserve the same, and simulations make it practical.

SOSam OkoroEngineering Manager, Agent Runtime · 6 min read

A one-line change to a playbook can change how an agent handles thousands of conversations. Most teams still check it by chatting with the agent a few times and hoping. Simulations replace hope with a repeatable rehearsal.

What a simulation is

A simulation plays a scripted customer against your agent, start to finish, with real actions pointed at test data. It checks the outcome you care about: was the refund issued, was the right policy cited, did the agent hand over when it should.

Build the suite from real conversations

  • Start with the twenty most common journeys, taken from real transcripts
  • Add every conversation that went wrong in the last month
  • Add the edge cases your team argues about

A suite of fifty scenarios covers more ground than a week of manual testing, and it runs in minutes.

Make it a gate, not a report

Run the suite on every change and block the release when a scenario fails. In Parley, simulations run from the CLI or on publish, and a failed scenario shows the trace of exactly where the conversation went another way.

Run the suite on every change and block the release when a scenario fails.

Pair it with split testing

Simulations tell you a change is safe. Split testing tells you it is better. Ship passing changes to a small share of real traffic first, compare resolution against the main version, then roll out.

Keep the suite alive

Every handoff that should not have happened is a new scenario. Teams that add one or two a week end up with a suite that knows their customers better than any single person on the team.

SOSam OkoroEngineering Manager, Agent Runtime, ParleyAll posts

Keep reading from the journal

All posts
Operations5 min read

One agent, every language: localising support without a second build

Read
Conversation design6 min read

What a good handoff from AI to a person actually looks like

Read
Measurement7 min read

Measure resolution, not deflection: a CX scorecard your finance team will trust

Read

Try it on your own conversations

Start building an agent in minutes, or talk to us about your support volume.

Get startedTalk to sales