Part 10

Putting it together: tuning playbooks + verify

Scenario-driven recipes that combine every knob from Parts 2–9, the dashboard→validator→runtime mapping discipline (including the dead-knob, env-gated, and default-mismatch traps), and the close-the-loop habit of changing a setting via the API and then observing the change on a real call.

Step 57

Scenario playbooks recipes

What it is. The individual knobs are only useful as a combination. Each deployment shape has a small set of settings that matter together. Pick the scenario that matches the customer's environment, set the cluster of values below, then verify (Step 60). Don't tune one knob at a time in isolation: noise, barge-in, and latency trade off against each other.

Noisy call center

Digit / TC / IBAN collection

Barge-in-heavy chatty caller

Latency-sensitive deployment

Speaker-mode / hands-free echo

Symptom these fix: "the bot keeps cutting itself off in the branch office", "it can't hear digits over the floor noise", "it talks over me", "there's a half-second of dead air every turn", "on speakerphone it interrupts itself."

Try it — tuning playbook selector

Pick a scenario, get the setting cluster + rationale
Step 58

The mapping recap: dashboard → validator → runtime discipline

What it is. The discipline that makes you trustworthy on a customer call: for any knob, you can name its config key, its validator range/default, and its pipecat-agent consumer (file:line). The full settings table is in phase0-report.md — the searchable finder below reproduces it. Three classes of trap to keep front of mind:

Dead knobs — do not promise a fix via these confidenceReask.* (the entire re-ask group), stopSpeakingPlan.voiceSeconds, stopSpeakingPlan.backOffSeconds, and aicEnabled (read but hard-disabled for "crate instability"). They exist in the UI/schema but no runtime consumer reads them.
Env-gated — present but conditional deepFilterEnabled does nothing without NC_OPT_URL pointed at the noise-cancel service. Provider / voice / language can be overridden by is_on_premise (on-prem forces FreyaTTS + Leyla regardless of the dashboard choice).
Default mismatches — schema says one thing, runtime another Runtime waitSeconds falls back to 0.6 (not the schema's 0.4). silenceTimeout defaults to 10 in the seed but 0 in the validator. voiceSeconds reads 0.3 in defaults vs 0.4 in the validator (and is dead anyway). If a customer's behavior doesn't match the dashboard number, suspect one of these.

The honest answer when you're unsure whether a knob is wired in a newer branch than dev is "verify with your pair" — not a guess.

Try it — setting → runtime mapping finder

Filter the full cross-part settings table (Parts 2–9)
0
Dashboard label Config key Validator range Default pipecat-agent file:line Status
Ask Claude Code: "For sttConfig.additionalSettings.vadStopSecs, find every place it is read in pipecat-agent, show me the file:line, and tell me the validator range and default from the dashboard schema — then tell me whether the runtime fallback matches the schema default."
Step 59

Changing a setting via the API GET → merge → PUT → publish

What it is. Settings live on the agent config. The safe pattern (the same one the team's MCP freya_agents workflow uses): GET the agent, merge your single change into the JSON, PUT the whole object back. Never blind-write a partial body — the response is double-nested and a naive write will silently wipe sibling fields (system_prompt has been lost this way). Then publish a version so the change actually goes live.

# 1. GET current agent config (note the double-nested response shape)
curl -s "$FREYA_API_URL/api/v2/agents/$AGENT_ID" \
  -H "Authorization: Bearer $FREYA_API_KEY" > agent.json

# 2. edit agent.json: set sttConfig.additionalSettings.vadStopSecs = 0.5
#    (merge into the existing object; keep every sibling field intact)

# 3. PUT the full object back
curl -X PUT "$FREYA_API_URL/api/v2/agents/$AGENT_ID" \
  -H "Authorization: Bearer $FREYA_API_KEY" \
  -H "Content-Type: application/json" \
  --data @agent.json

# 4. publish a new version so the change goes live
#    (use the dashboard "Publish" button, or the MCP freya_agents publish action)

Trap it avoids: "I changed it in the dashboard but the call still behaves the old way" — almost always an unpublished draft or a partial-write that dropped fields. Confirm the exact endpoint shape and publish step with your pair / the MCP freya_agents tool; response bodies are double-nested.

Ask Claude Code: "Read this agent's config via the freya_agents MCP tool, raise vadStopSecs to 0.5, PUT it back, publish a new version, then make a test call and pull ?track=user to confirm the early cutoff is gone."
Step 60

Verify the change with a real call or simulation close the loop

What it is. A tuning change isn't done until you've observed it. Three ways to close the loop, in increasing rigor:

  1. Test web/phone call. Reproduce the original symptom and confirm it's gone. For VAD / echo issues, pull the separated recording tracks — combined audio hides the gap:
    GET /api/v2/call/{callId}/recording?track=user ?track=assistant
    Measure the silence between the caller's last word (user track) and the bot's first word (assistant track). Without track= you get the mixed file and the gap is invisible.
  2. Simulation. For systematic changes, run the agent against synthetic personas in the simulation engine and compare pass rate before / after.
  3. Check the runtime logs. The agent logs the actual VAD params it booted with. If your dashboard change didn't reach this line, it didn't take effect — suspect a stale published version or an env override:
    VAD params: confidence=..., start_secs=..., user_turn_stop_timeout=...
    # base_service.py:1552

The habit that matters: the base_service.py:1552 log-line check is what separates "I changed the dashboard" from "the change is live." Always close the loop.

Ask Claude Code: "Pull the pipecat-agent logs for call {callId}, find the base_service.py:1552 VAD params line, and tell me whether user_turn_stop_timeout matches the vadStopSecs=0.5 I just published — if not, find which env override or stale version is winning."

Appendix — the dead-knob honesty list tell the truth

Tell customers the truth about what does nothing today (confirmed against pipecat-agent@dev). When you're unsure whether a knob is wired in a newer branch, say "(verify with your pair)" rather than guessing.

KnobStatus
confidenceReask.* (entire re-ask group)UI + schema present, no runtime consumer
stopSpeakingPlan.voiceSecondsvisible slider, no consumer
stopSpeakingPlan.backOffSecondshidden div, no consumer
aicEnabledread but hard-disabled ("crate instability")
Deepgram keywordsdisabled (TODO); only keyterm works