Decision #317AcceptedTrack · AI in Product3 min read

Your360 AI's Voice Coach Runs 44-Minute Calls on State Machines

Your360 AI replaced a single prompt with state machines, moved Retell to LiveKit, and runs 44-minute median debriefs — with two-pass evals guarding anonymity.

Building an AI Voice Agent for 360 Degree Feedback: How Your360 AI Uses State Machines, Evals, and Layered Synthesis
Building an AI Voice Agent for 360 Degree Feedback: How Your360 AI Uses State Machines, Evals, and Layered SynthesisAI-generated

Context

  1. Median report debrief call is 44 minutes; the first debrief broke a 60-minute platform limit

  2. Your360 AI moved from Retell AI to LiveKit to control Deepgram speech-to-text, the LLM, and ElevenLabs text-to-speech

  3. The self-discovery agent shipped and was validated a month before feedback collection existed

  4. Anonymity rules require corroboration for critical feedback and groups shown only at three or more providers

  5. Evals use personas built from failed calls to regression-test model and provider upgrades

By Priya Raman

A median 44-minute debrief call is the number Your360 AI uses to prove its core bet: people will talk to a voice agent about their own blind spots far longer than they will fill out a survey. The first report debrief call ran so long it broke the platform's 60-minute limit.

On the podcast Just Now Possible, Teresa Torres interviewed the three-person leadership team behind the product: Co-Founder and CEO Jared Goralnick, Head of Design Eda Goksel Jackson, and AI Engineer Alex Krasun. Their voice coach, Tam, runs three calls per 360 cycle: self-discovery, feedback collection, and the report debrief.

Why a single prompt failed

The team's first architecture was one prompt. It could not hold a structured conversation. They replaced it with state machines that track transitions between strengths, growth areas, and requested focus areas — while still letting the agent go deep when a respondent wants to.

Voice also collapses the survey's pacing assumptions. "People answer three questions in their first sentence," the team explained, so the agent has to track what has already been covered and decide when to push and when to move on. That state tracking, not naturalness, is the hard part.

Owning the voice stack

Your360 AI started on Retell AI, then moved to LiveKit, the open-source voice orchestration layer, to control every layer of a cascading stack — speech-to-text via Deepgram, the LLM, and text-to-speech via ElevenLabs.

The tradeoff they report: model personality versus time to first token. Reasoning models, which think before responding, get in the way of conversational flow. Latency beats depth in voice.

What do the evals look like?

The team built personas from failed calls and uses them to regression-test every model and provider upgrade. They sequence the work deliberately:

  • The self-discovery agent shipped and was validated a month before feedback collection existed
  • Synthesis came after that, built one call at a time
  • Observability runs on automated flags for sentiment, customer value, and product quality — humans step in only when something goes wrong

One early signal about the market: if a customer won't let you look at conversations in the early days, that tells you something. Their answer was to find design partners who would.

Synthesis requires two eval passes. Structural passes catch "don't mess up" errors such as misattributed feedback. Substantive passes re-run the analysis with few rules to surface patterns the first pass missed. Both matter; neither alone is sufficient.

How does anonymity survive synthesis?

The anonymity rules are concrete: critical feedback requires corroboration, and feedback groups are shown only when there are three or more providers. The debrief uses progressive reveal, so recipients process one theme at a time rather than confronting a wall of findings. Reports are delivered through each company's own values and leadership principles, and coach Rebecca Glenn, author of Fearless Feedback, helped shape the report format.

At the organizational level, Your360 AI generates heat maps, split between interdependent teams and independent cohorts — the aggregation lens a people-analytics team would ask for.

Usability testing a voice call required its own method: screen recordings paired with follow-up interviews that compare how the caller perceived the call with what actually happened.

What comes next

The roadmap targets the gap between a person's first and second 360 — turning awareness into action. Feedback collection calls are tuned for completion and low friction, including plain phone calls.

The broader signal for product teams: as voice agents move from demos to production, the winning stack is state machines over prompts, owned orchestration over managed platforms, and eval pipelines built from real failures — the same regression discipline software teams already apply to code.

The episode is available on Spotify and Apple Podcasts; full transcripts are limited to paid subscribers.

via open.spotify.com (Original)

More from Priya Raman

Priya Raman

Show full bio

News editor covering money, funding and market shifts around product management at Roadmap File.

34 articles