GitHub - collabora/WhisperSpeech: An Open Source text-to-speech system built by inverting Whisper.

If you have questions or you want to help you can find us in the #audio-generation channel on the LAION Discord server. An Open Source text-to-speech system built by inverting Whisper. Previously known as spear-tts-pytorch. We want this model to be like Stable Diffusion but for speech – both powerful and easily customizable. We are working only with properly licensed speech recordings and all the

[stealthinu]

WhisperのエンコーダのQuantizerとTransformerまで含めた（つまり意味理解まで含めた）ベクトルを使ってデコーダのVocosへ与えるネットワーク組んでTTSを再構築している。頭いい。