hyperframes-media

Generate speech, transcribe audio with timestamps, and remove video backgrounds for transparent overlays.

- Three CLI commands (tts, transcribe, remove-background) that each download and cache their own model on first run; no API keys required

- Text-to-speech supports 54 multilingual voices (American, British, Spanish, French, Hindi, Italian, Japanese, Portuguese, Mandarin) with speed control; auto-detects language from voice prefix

- Transcription produces word-level timestamps in normalized JSON; supports multiple input formats (audio, video, SRT/VTT, OpenAI responses) with configurable Whisper model sizes and explicit language selection to prevent silent translation errors

- Background removal outputs VP9 WebM with alpha channel (or ProRes/PNG) for transparent overlays; optional --background-output flag creates a hole-cut inverse layer for compositing text or graphics between subject and background

HyperFrames Media

CLI commands that create assets (tts, bgm, transcribe, remove-background), plus everything needed to consume and animate transcript data in HTML. For placing assets into compositions, see hyperframes-core.

Provider chains (auto-detected from env)

TTS - npx hyperframes tts "..." picks the first available provider:

Order
Provider
Detected when
Word timestamps

1
HeyGen (Starfish)
$HEYGEN_API_KEY / hyperframes auth login
Yes, native - pass --words narration.words.json to capture

2
ElevenLabs
$ELEVENLABS_API_KEY set
No - chain transcribe after

3
Kokoro-82M (local, 54 voices)
always (no key required)
No - chain transcribe after

If the installed hyperframes tts is the local-only build (its --help says "Kokoro-82M" and has no --provider/--words flags), it silently falls back to Kokoro even with $HEYGEN_API_KEY set. To force HeyGen regardless of CLI version, use the self-contained scripts/heygen-tts.mjs (see references/tts.md).

BGM - npx hyperframes bgm --duration N: