Back: Catalog

WhisperSpeech

WhisperSpeech is an open-source text-sound (TTS) system created by the reversal of the OpenAI Whisper model. Architecturally, it follows a two-stage token-based pipeline; Whisper for transcription, EnCodec for waveform tokenization, and Vocos for high fidelity sound. Available with Apache-2.0 and MIT licenses, the models are trained with licensed data and feature optimizations that can exceed real-time speed over RTX 4090.

WH

WhisperSpeech

62

collabora · Voice & Text-to-Speech

STARS4
FORKS271
COMMITS264
CONTRIBUTORS9
VERSION—
UPDATED10 mo ago
JupyterWhisper-BasedStreamingSes TokenizerMIT

Common commercial products in this category: ElevenLabs, Amazon Polly, Google Cloud Text-to-Speech

What can it do?
  • You make fluent transitions between different languages in a single sentence
  • With one click, you perform voice-cloning over existing examples.
  • Using the Colab environment or running it quickly with pre-trained models on your local computer
User reviews
  • ↑ PositiveHacker News

    WhisperSpeech – An open source text-to-speech system built by inverting Whisper

    “Yeah, thanks. I'd love to try to clarify this (I'll put this into our documentation as well ASAP) for anyone that may be reading this in the future: Our model is not a derivative of Whisper but we do use the transcripts (and encoder outputs) in our data preprocessing. I am…”

Website GitHub

My note

Alternative pages

ElevenLabs alternativesAmazon Polly alternativesGoogle Cloud Text-to-Speech alternatives

More in: Voice & Text-to-Speech