Skip to content

Conversation

@mitulkhemani2005
Copy link

Interrupt Words Handling Logic

Introduction
I am Mitul Khemani, a third-year B.Tech student at LNMIIT Jaipur.
Roll Number: 23UEC572

Summary
This PR implements a context content aware backchannel filtering system that prevent's the AI voice agent from being interrupted by passive acknowledgement words like ("yeah", "ok", "hmm") while agent is speaking, while still allowing real interrupting words like ("stop", "wait") to interrupt immediately and stop the AI voice agent.

Problem
When user listen to an AI agent, they often provide verbal feedback like "yeah", "ok", "hmm" to indicate they are engaged and listening to the AI agent but LiveKit's default Voice Activity Detection (VAD) interprets the agent thinking as interruptions causing the agent to stop speaking on the spot.

Solution
A state aware logic layer was added that distinguishes between passive acknowledgements and active interruptions based on:

  1. Whether the agent is currently speaking or silent
  2. Whether the transcript contains inttruption words or not

Person can change the backchannel filter words from livekit-agents/livekit/agents/voice/backchannel_filter.py

Person can run the agent by running python ./examples/voice_agents/basic agent.py cosole or python ./examples/voice_agents/basic agent.py dev depends on the usage mode.
before runing please update.env files

Person can run all backchannel filter test using python -m pytest tests/test_backchannel_filter.py -v

Demo Video and Transcript
Video: Video Link
Transcipt: Transcript Link

Scenario Tested

Scenario Test Status
The Long Explanation Agent reading a paragraph, user says "okay... yeah... uh-huh" ✅ Agent continues without breaking
The Passive Affirmation Agent asks "Are you ready?", goes silent, user says "Yeah" ✅ Agent processes as answer and responds
The Correction Agent counting "1, 2, 3...", user says "No stop" ✅ Agent cuts off immediately
The Mixed Input Agent speaking, user says "Yeah okay but wait" ✅ Agent stops (contains "wait" command)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant