Inference fails with CUDA out of memory on long scripts

[rachana192837]

I tried to run the 1.5B multi-speaker model on a 12GB GPU using a script longer than 10 minutes.

Expected behavior:

• The model generates audio for the entire script.

Actual behavior:

• The process crashes with a CUDA Out of Memory (OOM) error after a few minutes.

Steps to reproduce:

1. Clone the VibeVoice repository.
2. Install dependencies as per the instructions.
3. Run inference with the 1.5B multi-speaker model on a script longer than 10 minutes.

Environment:

• OS: Ubuntu 22.04
• Python: 3.11
• CUDA: 12.1
• GPU: NVIDIA RTX 3060 12GB
• VibeVoice model: 1.5B multi-speaker

Additional notes:

• Reducing the script length allows the inference to succeed.
• Suggestion: maybe implement memory optimization for long-form audio generation.

[zshnb]

same environment and same behavior, it use too much gpu memory than other asr model

[rachana192837]

Thanks for the feedback
I’m observing the same behavior on a similar setup.
I’ve opened PR #158, which adds a chunked inference approach for long-form scripts. This significantly reduces peak GPU memory usage and avoids CUDA OOM during long-form generation, without changing model weights or architecture.