Implemented Interrupt Words Handling Logic by Mitul Khemani #479
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Interrupt Words Handling Logic
Introduction
I am Mitul Khemani, a third-year B.Tech student at LNMIIT Jaipur.
Roll Number: 23UEC572
Summary
This PR implements a context content aware backchannel filtering system that prevent's the AI voice agent from being interrupted by passive acknowledgement words like ("yeah", "ok", "hmm") while agent is speaking, while still allowing real interrupting words like ("stop", "wait") to interrupt immediately and stop the AI voice agent.
Problem
When user listen to an AI agent, they often provide verbal feedback like "yeah", "ok", "hmm" to indicate they are engaged and listening to the AI agent but LiveKit's default Voice Activity Detection (VAD) interprets the agent thinking as interruptions causing the agent to stop speaking on the spot.
Solution
A state aware logic layer was added that distinguishes between passive acknowledgements and active interruptions based on:
Person can change the backchannel filter words from
livekit-agents/livekit/agents/voice/backchannel_filter.pyPerson can run the agent by running
python ./examples/voice_agents/basic agent.py cosoleorpython ./examples/voice_agents/basic agent.py devdepends on the usage mode.before runing please update.env files
Person can run all backchannel filter test using
python -m pytest tests/test_backchannel_filter.py -vDemo Video and Transcript
Video: Video Link
Transcipt: Transcript Link
Scenario Tested