VPS Tutorials
Self-host Whisper transcription: Speaches, models and subtitles
Compare openai-whisper, faster-whisper and whisper.cpp, size CPU/GPU resources, and deploy a private Speaches audio API. Follow batch transcription, subtitle generation, ffmpeg preprocessing and Open WebUI integration, with explicit Chinese-audio examples and current configuration caveats.

Whisper can turn recordings, interviews and videos into text on infrastructure you control. A private service can support batches and integrations, but compute, storage, transfer and maintenance still cost money. Verify where audio is processed and stored; self-hosting on rented infrastructure does not remove provider or administrator trust requirements.
This guide compares implementations and models, deploys Speaches, produces subtitles and connects an application. It preserves Chinese-audio examples as language-specific cases rather than assuming every reader transcribes Chinese.
| Implementation | Stack | Useful considerations |
|---|---|---|
| openai-whisper | Reference Python implementation | Broad reference compatibility; GPU/CPU and timestamp options depend on settings |
| faster-whisper | Python and CTranslate2 | Efficient inference, CPU/GPU quantization and optional word timestamps |
| whisper.cpp | C/C++ | CPU/ARM/Apple and other supported backends; quantization/acceleration vary |
The original cited roughly 4× faster inference, RTF 0.136 and WER 3.12% versus 26.92%. These are benchmark-specific figures without a shared verified setup here, not universal speed/accuracy rankings. Do not assume whisper.cpp is always fastest or necessarily inaccurate, or that the reference implementation is obsolete.
- NVIDIA server: test faster-whisper with compatible CUDA/cuDNN.
- CPU/ARM/local device: compare whisper.cpp and faster-whisper CPU settings.
- Need reference behavior/dependencies: openai-whisper remains an option.
WhisperX can add alignment/diarization using separate models and permissions; faster-whisper alone does not identify speakers automatically. Timestamp capabilities also depend on implementation/options.
| Model | Original approximate download | Indicative VRAM | Planning use |
|---|---|---|---|
| tiny | ~75 MB | <1 GB | Quick trials; accuracy can be limited |
| base | ~150 MB | ~1 GB | Small workloads where measured quality is sufficient |
| small | ~500 MB | ~2 GB | Useful first benchmark |
| medium | ~1.5 GB | ~5 GB | More capacity for difficult audio |
| large-v3 | ~3 GB | ~6 GB | Higher-quality candidate with larger resources |
Converted/quantized artifacts, batching and context change disk/VRAM use. For languages other than English choose a suitable multilingual artifact, not an English-only model.
- For Chinese audio, medium is a useful starting comparison; evaluate terminology and punctuation.
- For clear English, small or base may suffice after testing.
- For published subtitles, compare large-v3 and proofread; no model guarantees correct captions.
- For many files, use small as a first pass and rerun weak segments with a larger model.
Persist the cache; switching may reuse downloaded files, but missing/revised artifacts still download. The original's roughly 6 GB for a set of models is only an example.
Original small-model examples estimated 4-core CPU at 1–3× real time (20–60 minutes per hour of audio), GTX 1660-class GPU at 10–20× (3–6 minutes), and a 5–10× GPU advantage. These are workload-dependent estimates, not measurements of a provider or promises.
| Workload | Initial planning budget |
|---|---|
| Occasional short files | About 2 cores/4 GB, CPU and small model |
| Frequent long batches | Suitable GPU with around 6 GB+ VRAM; validate large-model settings |
| Real-time use | GPU budget around 8 GB+ plus tested streaming latency |
Decoding consumes system RAM too: consider 4 GB for small CPU trials, 8 GB or more for concurrent work. See GPU sizing. Original cache examples were 5 GB for small/medium and 15 GB for a broader set; include OS, decoded audio, output and container layers beyond those budgets.
Speaches wraps speech models behind an OpenAI-compatible API. Compatibility is a supported subset, not a guarantee every OpenAI audio/realtime feature or every application works unchanged. Verify your version's schema, model names and response formats.
CPU docker-compose.yml (adapt the legacy TTL line as explained below before startup):
services:
speaches:
image: ghcr.io/speaches-ai/speaches:latest-cpu
container_name: speaches
restart: unless-stopped
ports:
- "127.0.0.1:8969:8000"
environment:
- API_KEY=换成一串长随机字符
- PRELOAD_MODELS=["Systran/faster-whisper-small"]
- WHISPER__TTL=-1
volumes:
- ./hf-cache:/home/ubuntu/.cache/huggingface/hub
GPU variant, merged into the same service definition:
image: ghcr.io/speaches-ai/speaches:latest-cuda
deploy:
resources:
reservations:
devices:
- driver: nvidia
count: 1
capabilities: [gpu]
Create the project directory first, choose maintained compatible image tags/digests and ensure effective cache UID/path match the mounted directory. Follow the GPU toolkit procedure and test the application's actual driver/runtime, not merely a different CUDA test image.
Set private API_KEY and verify authentication on /v1 with Authorization: Bearer 你的密钥. Inspect any health/docs/model-management exceptions instead of assuming all routes share the same policy. In the current configuration, /health, /docs, /openapi.json and the Gradio interface are exceptions. Set ENABLE_UI=false if you only need the API, and keep the whole service behind the private boundary; an API key alone does not protect the default UI.
PRELOAD_MODELS downloads the configured model files during startup. It does not guarantee that inference is already warmed up; the first transcription can still load a model into memory. WHISPER__TTL=-1 is a legacy environment spelling: the current official configuration uses STT_MODEL_TTL=-1 to keep speech-to-text models resident. Replace the legacy line in the example before launch; use a finite supported TTL when memory is limited. Persist cache to avoid unnecessary downloads; do not expose port 8969 directly. Remote access needs a private network/tunnel or authenticated HTTPS proxy.
docker compose up -d
docker compose logs -f
Application startup complete confirms application startup, not successful model inference. Test one authorized recording, inspect logs and verify output before processing a large queue.
Run the loopback example on the VPS or its configured tunnel:
curl -s http://127.0.0.1:8969/v1/audio/transcriptions \
-H "Authorization: Bearer 你的API_KEY" \
-F "[email protected]" \
-F "model=Systran/faster-whisper-small" \
-F "language=zh"
language=zh explicitly selects Chinese for this example; use the actual audio language or supported detection for other material. A known language can help, but does not guarantee accuracy.
The batch example requires Python and requests, an audios directory and the correct private key:
import os, requests
API = "http://127.0.0.1:8969/v1/audio/transcriptions"
KEY = "你的API_KEY"
HEADERS = {"Authorization": f"Bearer {KEY}"}
for name in os.listdir("audios"):
if not name.endswith((".mp3", ".wav", ".m4a", ".flac")):
continue
with open(f"audios/{name}", "rb") as f:
r = requests.post(
API,
headers=HEADERS,
files={"file": f},
data={
"model": "Systran/faster-whisper-small",
"language": "zh",
"response_format": "srt",
},
timeout=3600,
)
out = os.path.splitext(name)[0] + ".srt"
with open(out, "w", encoding="utf-8") as f:
f.write(r.text)
print("完成:", out)
Before production use, call r.raise_for_status() immediately after the request, before opening the output file. This prevents error JSON being written as SRT. Also protect keys, keep output names unique, and record completed inputs for restart. A 3600-second client timeout is a starting setting; long jobs/proxies may need different limits and bounded retries.
response_format can select supported json, text, srt, vtt or verbose_json. Use srt for compatible subtitle output and inspect timestamps. Formats/options vary by model and server version.
ffmpeg -i video.mp4 -vn -ac 1 -ar 16000 -c:a libmp3lame -b:a 64k audio.mp3
-vn removes video, -ac 1 mixes to mono and -ar 16000 resamples to 16 kHz. Whisper processes 16 kHz audio, but the example's lossy MP3 encoding can alter recognition and mono mixing loses channel separation. For mono lossless PCM, use ffmpeg -i video.mp4 -vn -ac 1 -ar 16000 -c:a pcm_s16le audio.wav. Retain separate channels when speaker separation matters. A 1 GB video becoming 20 MB audio is duration/codec-dependent, not guaranteed compression.
Validate SRT in a player or burn it in with ffmpeg:
ffmpeg -i video.mp4 -vf subtitles=audio.srt -c:a copy output.mp4
This re-encodes video; retain the original and check subtitle timing, font support and output size.
For an existing Open WebUI setup, use supported speech-to-text settings:
AUDIO_STT_ENGINE=openai
AUDIO_STT_OPENAI_API_BASE_URL=http://speaches:8000/v1
AUDIO_STT_OPENAI_API_KEY=你的API_KEY
AUDIO_STT_MODEL=Systran/faster-whisper-small
API_BASE_URL must be reachable from the component sending audio. When both services share Docker's network, use the Speaches service hostname/8000; 127.0.0.1:8969 points to a different loopback context inside another container. Check the installed application's configuration names and verify it sends speech to your chosen endpoint. Other compatible apps still need feature-specific testing.
Traditional/simplified mixture can be normalized with OpenCC:
pip install opencc-python-reimplemented
# 代码里用 OpenCC('t2s').convert(text) 转简体
Choose conversion appropriate to the material, retaining the original transcript. Normalization does not repair recognition errors or guarantee regional terminology.
Use prompt as contextual guidance where supported:
-F "prompt=本次会议讨论:Kubernetes、PostgreSQL、张伟、李娜"
The glossary example concerns Chinese names and technical terms; supply your actual vocabulary. Prompts can help but also bias output and do not guarantee correct names. Review punctuation manually or with controlled post-processing rather than assuming a script can infer every missing mark.
Overlapping speech/background music remains difficult. Diarization needs a separate compatible workflow/model, potentially with account/license requirements; it is not included just because Speaches uses faster-whisper.
For occasional recordings, compare a current cloud quote with hosting/time instead of treating the source's cents-to-over-a-dollar-per-hour estimate as universal pricing. For local-only use, clients such as MacWhisper or Buzz may avoid server administration, though hardware/software terms still apply.
Server deployment suits multi-device access, repeated batches, automation and a controlled processing network. Real-time subtitles require a tested streaming client/server path and continuous GPU capacity; an OpenAI-compatible transcription endpoint alone does not establish realtime compatibility.
VAD can skip silence and reduce hallucinated silent-segment text, but thresholds can also drop quiet speech. The source's 20–30% silence estimate is illustrative; validate on your recordings and preserve original audio.
For supported compute_type:
| Setting | Typical target | Tradeoff |
|---|---|---|
float16 | Compatible GPU | Lower memory/compute cost than full precision; hardware dependent |
int8_float16 | Supported GPU under VRAM pressure | Quantization saves memory; quality/support vary |
int8 | CPU where supported | Lower memory; benchmark actual speed |
int8_float16 may help a large model fit around 4 GB under some settings, without a general guarantee. Check CTranslate2 support before forcing it. In the current Speaches configuration this is WHISPER__COMPUTE_TYPE, for example WHISPER__COMPUTE_TYPE=int8 for a compatible CPU setup.
Use ffmpeg to segment a long recording when appropriate; 3 hours split into roughly 10-minute pieces is a strategy example. Preserve boundary context and offset/rejoin subtitle timestamps. More parallel requests can exhaust RAM/VRAM or serialize internally; benchmark bounded concurrency, not simply the CPU core count. Small-first/larger-second processing can save work, without a guaranteed order-of-magnitude speedup.
First call is slow: inspect downloads/loading and PRELOAD_MODELS; distinguish readiness from inference.
404 model not found: inspect supported names using curl http://127.0.0.1:8969/v1/models -H "Authorization: Bearer 密钥". POST /v1/models/{模型名} downloads a supported model in the current API. The full model ID may contain a slash; use the path supported by your installed schema. /v1/models lists downloaded models, whereas /v1/registry lists supported remote models. A successful download does not prove GPU inference is ready.
GPU unused: check nvidia-smi, toolkit, runtime and selected compute settings; a visible GPU alone does not prove inference uses it.
Repeated words/hallucinations: inspect silence/music/language detection, try appropriate VAD and ffmpeg preprocessing, and compare models; a larger model is not a complete fix.
Memory exhausted: review the legacy WHISPER__TTL=-1 against the current STT_MODEL_TTL setting, simultaneous resident models, batch size and request limits.
Too many jobs: use bounded queues/rate limits and tested timeouts, rather than exposing unlimited parallel inference.
Choose implementation and model from an actual sample of your recordings, deploy a private endpoint, validate output and then add automation. Keep original audio and reviewed transcripts; processing control and repeatable batches are the useful gains.