Skip to content

Add intelligent interruption handling#489

Open
fxhxdxd wants to merge 1 commit intoDark-Sys-Jenkins:mainfrom
fxhxdxd:feature/interrupt-handler-fahad-khan
Open

Add intelligent interruption handling#489
fxhxdxd wants to merge 1 commit intoDark-Sys-Jenkins:mainfrom
fxhxdxd:feature/interrupt-handler-fahad-khan

Conversation

@fxhxdxd
Copy link

@fxhxdxd fxhxdxd commented Feb 2, 2026

What's added

  • Backchannel-aware interruption logic for LiveKit voice agents.
  • Configurable vocab for backchannels and interrupt words.
  • Example agent demonstrating the behavior.

What was the challenge

  • Default VAD pauses on filler words ("yeah/ok/hmm") while the agent is speaking, causing hiccups.
  • We need to ignore passive acknowledgements during agent speech but still interrupt on real commands or mixed input.

Why it works

  • State-aware filter (BackchannelFilter) plus gating in agent_activity.py: only pause speech when STT confirms the user said something other than a pure backchannel.
  • Backchannels while speaking are ignored; commands or mixed phrases interrupt; when silent, all input (including "yeah") is processed.
  • Fast path on interim transcripts for interrupt words; otherwise defer to final STT to avoid stutters.
  • Vocab is configurable via AgentSession(backchannel_words=..., interrupt_words=...) or env vars LIVEKIT_BACKCHANNEL_WORDS / LIVEKIT_INTERRUPT_WORDS.

How to run (console)

  1. Ensure env keys (As .env.local) for your chosen STT/TTS/LLM and LiveKit (LIVEKIT_URL, LIVEKIT_API_KEY, LIVEKIT_API_SECRET).
  2. uv run python examples/voice_agents/intelligent_interruption.py console
  3. While the agent speaks, say "yeah/ok/uh-huh" → it keeps talking. When silent, say "yeah" → it responds. Say "stop" or "yeah wait" → it interrupts.

How to run (dev with Agents Playground)

  1. Set env (As .env.local) : LIVEKIT_URL, LIVEKIT_API_KEY, LIVEKIT_API_SECRET.
  2. Start : uv run python examples/voice_agents/intelligent_interruption.py dev.
  3. Open https://agents-playground.livekit.io, connect and talk to the agent to verify the backchannel behavior.

Proof

Screenshot 2026-02-02 at 8 23 22 PM Screenshot 2026-02-02 at 8 23 38 PM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant