How to Run VibeVoice-ASR-Streaming Locally

01

What you need

Microsoft recommends an NVIDIA Deep Learning Container to manage the CUDA environment. Its current guide marks NVIDIA PyTorch containers 24.07 through 25.12 as verified and uses nvcr.io/nvidia/pytorch:25.12-py3 in the example.

You also need the VibeVoice repository, Python tooling inside the container and FFmpeg for audio decoding. If FlashAttention is not already available, Microsoft points to installing flash-attn separately.

02

Install VibeVoice and FFmpeg

Inside the GPU-enabled environment, Microsoft's documented flow is to clone https://github.com/microsoft/VibeVoice.git, change into the VibeVoice directory and run pip install -e .

Install FFmpeg with apt update && apt install ffmpeg -y. Both documented usage paths rely on FFmpeg for audio decoding, so verify it before assuming an audio failure is a model failure.

03

Run the FastAPI microphone demo

Microsoft provides demo/vibevoice_asr_streaming_fastapi_demo.py with a model_path argument. After starting it, open http://localhost:7870 to record from a microphone or select a file.

The page keeps a WebSocket open for the entire recording session, allowing transcript text to appear while speech is still arriving. This is the simplest end-to-end check that the checkpoint, audio decoder and streaming UI path work together.

04

Test file streaming and hotwords

For a non-browser test, Microsoft documents demo/vibevoice_asr_streaming_inference_from_file.py with model_path and audio_files arguments. Each chunk is printed as soon as the model emits it.

Add context_info such as Microsoft,VibeVoice to bias recognition toward important names or technical terms. Treat hotwords as guidance rather than a guarantee and test them against representative audio.

Microsoft says chunk size and lookahead are read from the checkpoint's preprocessor_config.json. Measure end-to-end latency on your own hardware instead of inventing a universal streaming-delay figure.

05

Move from demo to a service carefully

The repository links a vLLM streaming serving path, but a self-hosted service still needs session lifecycle handling, queueing, retries, observability, GPU-memory controls and explicit policies for storing or discarding audio and transcripts.

Local inference can reduce the need to send raw audio to a third-party transcription API, but local does not automatically mean compliant. Consent, access controls, retention rules and security remain your responsibility.

06

Troubleshooting checklist

• Confirm the NVIDIA GPU is visible inside the container
• Check PyTorch/CUDA and FlashAttention compatibility
• Verify FFmpeg and the input audio format
• Confirm the official checkpoint path and exact model ID
• Profile buffering, WebSocket/UI delay and model inference separately
• Test speaker attribution and hotwords on representative recordings before production use

Do not assume the streaming model supports the same 50-plus languages as the earlier batch VibeVoice-ASR. The official streaming model cards currently list ten languages.