AI 语音开源项目推荐(132 个)

统一文本、视觉、音频与多模态模型的定义框架

输入主题或关键词，自动生成高清短视频

本地运行与训练大模型和扩散模型的桌面应用

本地运行的开源语音克隆与配音工作台

开源智能体视频制作系统，让 AI 编程助手变身视频工作室

少样本语音克隆与文本转语音的 WebUI 工具

5 秒音频克隆声音，实时生成任意语音

OpenAI Whisper 语音识别模型的 C/C++ 高性能推理实现

开源本地 AI 引擎，无需 GPU 即可运行多种模型

🐸💬 - a deep learning toolkit for Text-to-Speech, battle-tested in research and production

A generative speech model for daily dialogue.

VoxCPM2: Tokenizer-Free TTS for Multilingual Speech Generation, Creative Voice Design, and True-to-Life Cloning

Instant voice cloning by MIT and MyShell. Audio foundation model.

🚀Clone a voice in 5 seconds to generate arbitrary speech in real-time

SOTA Open Source TTS

🚀 AI 全自动短视频引擎 | AI Fully Automated Short Video Engine

Mastra is the modern TypeScript framework for AI-powered applications and agents.

Faster Whisper transcription with CTranslate2

Readest is a modern, feature-rich ebook reader designed for avid readers offering seamless cross-platform access, powerful tools, and an intuitive interface to elevate your reading experience.

WhisperX: Automatic Speech Recognition with Word-level Timestamps (& Diarization)

统一文本、视觉、音频与多模态模型的定义框架

输入主题或关键词，自动生成高清短视频

本地运行与训练大模型和扩散模型的桌面应用

本地运行的开源语音克隆与配音工作台

开源智能体视频制作系统，让 AI 编程助手变身视频工作室

少样本语音克隆与文本转语音的 WebUI 工具

5 秒音频克隆声音，实时生成任意语音

OpenAI Whisper 语音识别模型的 C/C++ 高性能推理实现

开源本地 AI 引擎，无需 GPU 即可运行多种模型

🐸💬 - a deep learning toolkit for Text-to-Speech, battle-tested in research and production

A generative speech model for daily dialogue.

VoxCPM2: Tokenizer-Free TTS for Multilingual Speech Generation, Creative Voice Design, and True-to-Life Cloning

Instant voice cloning by MIT and MyShell. Audio foundation model.

🚀Clone a voice in 5 seconds to generate arbitrary speech in real-time

SOTA Open Source TTS

🚀 AI 全自动短视频引擎 | AI Fully Automated Short Video Engine

Mastra is the modern TypeScript framework for AI-powered applications and agents.

Faster Whisper transcription with CTranslate2

Readest is a modern, feature-rich ebook reader designed for avid readers offering seamless cross-platform access, powerful tools, and an intuitive interface to elevate your reading experience.

WhisperX: Automatic Speech Recognition with Word-level Timestamps (& Diarization)

An Industrial-Level Controllable and Efficient Zero-Shot Text-To-Speech System

Multi-lingual large voice generation model, providing inference, training and deployment full-stack ability.

Open-source speech recognition toolkit for training, inference, streaming ASR, VAD, punctuation, speaker diarization pipelines, and OpenAI-compatible/MCP serving.

Generate audiobooks from e-books, voice cloning & 1158+ languages!

A TTS model capable of generating ultra-realistic dialogue in one pass.

Translate the video from one language to another and embed dubbing & subtitles.

A scalable generative AI framework built for researchers and developers working on Large Language Models, Multimodal, and Speech AI (Automatic Speech Recognition and Text-to-Speech)

🧠 Leon is your open-source personal assistant.

kaldi-asr/kaldi is the official location of the Kaldi project.

Speech-to-text, text-to-speech, speaker diarization, speech enhancement, source separation, and VAD using next-gen Kaldi with onnxruntime without Internet connection. Support embedded systems, Android, iOS, HarmonyOS, Raspberry Pi, RISC-V, RK NPU, Axera NPU, Ascend NPU, x86_64 servers, websocket server/client, support 12 programming languages

Offline speech recognition API for Android, iOS, Raspberry Pi and servers with Python, Java, C# and Node

State-of-the-Art Deep Learning scripts organized by models - easy to train and deploy with reproducible accuracy and performance on enterprise-grade infrastructure.

Formerly KrillinAI, VideoTranslation and Dubbing Tool. Open-source AI workspace for creators, powered by Codex. Video downloading, video translation, create videos, images, voice, avatars, and edits with Agents in one place. Suitable for your Social Media like YouTube, TikTok, Instagram, etc.

Gradio WebUI for creators and developers, featuring key TTS (Edge-TTS, kokoro) and zero-shot Voice Cloning (E2 & F5-TTS, CosyVoice), with Whisper audio processing, YouTube download, Demucs vocal isolation, and multilingual translation.

Drench yourself in Deep Learning, Reinforcement Learning, Machine Learning, Computer Vision, and NLP by learning from these exciting lectures!!

Easy-to-use Speech Toolkit including Self-Supervised Learning model, SOTA/Streaming ASR with punctuation, Streaming TTS with text frontend, Speaker Verification System, End-to-End Speech Translation and Keyword Spotting. Won NAACL2022 Best Demo Award.

Use Microsoft Edge's online text-to-speech service from Python WITHOUT needing Microsoft Edge

A PyTorch-based Speech Toolkit

Very low latency speech to text, intent recognition, and text to speech, for building voice agents and interfaces

Real-time, local speech-to-text with streaming ASR, speaker diarization, translation, and OpenAI/Deepgram-compatible APIs.

OpenVINO™ is an open source toolkit for optimizing and deploying AI inference

AI Agent Engineering Platform built on an Open Source TypeScript AI Agent Framework

Amphion (/æmˈfaɪən/) is a toolkit for Audio, Music, and Speech Generation. Its purpose is to support reproducible research and help junior researchers and engineers get started in the field of audio, music, and speech generation research and development.

:robot: :speech_balloon: Deep learning for Text to Speech (Discussion forum: https://discourse.mozilla.org/c/tts)

End-to-End Speech Processing Toolkit

Open-source SenseVoiceSmall model for Mandarin, Cantonese, English, Japanese, and Korean ASR, language ID, emotion recognition, and audio event detection.

Speech recognition module for Python, supporting several engines and APIs, online and offline.

vits2 backbone with multilingual-bert

Generate audiobooks from e-books

EmotiVoice 😊: a Multi-Voice and Prompt-Controlled TTS Engine

A Deep-Learning-Based Chinese Speech Recognition System 基于深度学习的中文语音识别系统

Open Vision Agents by Stream. Build voice and vision agents quickly with any model or video provider. Uses Stream's edge network for ultra-low latency.

A text-to-speech (TTS), speech-to-text (STT) and speech-to-speech (STS) library built on Apple's MLX framework, providing efficient speech analysis on Apple Silicon.

VITS: Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speech

High-quality multi-lingual text-to-speech library by MyShell.ai. Support English, Spanish, French, Chinese, Japanese and Korean.

一个简单的本地网页界面，使用ChatTTS将文字合成为语音，同时支持对外提供API接口。A simple native web interface that uses ChatTTS to synthesize text into speech, along with support for external API interfaces.

ODS V3 Pre-Release: Public testing and refinement ahead of the official V3 launch. Turn your PC, Mac, or Linux box into a private AI server.

🤖 wukong-robot 是一个简单、灵活、优雅的中文语音对话机器人/智能音箱项目，支持ChatGPT多轮对话能力，还可能是首个支持脑机交互的开源智能音箱项目。

eSpeak NG is an open source speech synthesizer that supports more than hundred languages and accents.

💬 Speech recognition for your site

Facebook AI Research's Automatic Speech Recognition Toolkit

On-device Speech AI for Apple Silicon

FunASR-powered video transcription, subtitle generation, and LLM-assisted clipping tool with a local Gradio UI.

This repository contains a hand-curated resources for Prompt Engineering with a focus on Generative Pre-trained Transformer (GPT), ChatGPT, PaLM etc

StyleTTS 2: Towards Human-Level Text-to-Speech through Style Diffusion and Adversarial Training with Large Speech Language Models

Towards Human-Sounding Speech

A general-purpose AIGC video engine: script to finished film in one pipeline — dramas, ads, product videos, otome games, and more. | 通用 AIGC 视频引擎 —— 从剧本到成片一条流水线，漫剧、广告、电商、乙游皆可

All-in-One Development Tool based on PaddlePaddle

Silero Models: pre-trained text-to-speech models made embarrassingly simple

Generate audiobooks from EPUBs, PDFs and text with synchronized captions.

🎤 微软语音合成工具，使用 Electron + Vue + ElementPlus + Vite 构建。

Open source voice AI platform. Self-hosted alternative to Vapi and Retell. On Prem, BYOK across Speech to Speech or LLM/STT/TTS, with a visual workflow builder, MCP native and telephony support.

Open-source AI video localization and dubbing for YouTube/Bilibili: speech recognition, subtitle translation, voice cloning, audio mixing and rendering. 开源 AI 视频翻译配音工具。

Automatic Speech Recognition with Speaker Diarization based on OpenAI Whisper

Dockerized OpenAI-compatible wrapper for Kokoro-82M text-to-speech w/multiplatform CPU, AMD, NVIDIA GPU PyTorch; multi-speaker, clone-tuning, caption timestamps, SSML, optional readalong web UI

Production First and Production Ready End-to-End Speech Recognition Toolkit

Machine Learning and Agentic AI Resources, Practice and Research

On-device wake word detection powered by deep learning

DiffSinger: Singing Voice Synthesis via Shallow Diffusion Mechanism (SVS & TTS); AAAI 2022; Official code

Voice Recognition to Text Tool / 一个离线运行的本地音视频转字幕工具，输出json、srt字幕、纯文字格式

JAX implementation of OpenAI's Whisper model for up to 70x speed-up on TPU.

An Open Source text-to-speech system built by inverting Whisper.

The python library for real-time communication

这是一个Android系统TTS应用，内置微软演示接口，可自定义HTTP请求，可导入其他本地TTS引擎，以及根据中文双引号的简单旁白/对话识别朗读 ，还有自动重试，备用配置，文本替换等更多功能。

A 100M-parameter multilingual TTS model for real-time CPU inference, voice cloning, and 48 kHz stereo generation

A small speech recognizer

A nearly-live implementation of OpenAI's Whisper.

An open-source model family for long-form speech, dialogue synthesis, voice design, sound effects, and real-time streaming TTS

Foundational model for human-like, expressive TTS

Distilled variant of Whisper for speech recognition. 6x faster, 50% smaller, within 1% word error rate.

Converts text to speech in realtime

:stuck_out_tongue_closed_eyes: TensorFlowTTS: Real-Time State-of-the-art Speech Synthesis for Tensorflow 2 (supported including English, French, Korean, Chinese, German and Easy to adapt for other languages)

A simple, high-quality voice conversion tool focused on ease of use and performance.

An all-in-one, pure C++ inference engine for audio models, powered by ggml. Supports TTS, STT, VAD, voice conversion, music generation, and more, with highly optimized performance. No Python dependency.

OpenAI Whisper ASR Webservice API

A single Gradio + React WebUI with extensions for ACE-Step, OmniVoice, Kimi Audio, Piper TTS, GPT-SoVITS, CosyVoice, XTTSv2, DIA, Kokoro, OpenVoice, ParlerTTS, Stable Audio, MMS, StyleTTS2, MAGNet, AudioGen, MusicGen, Tortoise, RVC, Vocos, Demucs, SeamlessM4T, and Bark!

Whisper & Faster-Whisper standalone executables for those who don't want to bother with Python.

The official Python SDK for the ElevenLabs API.

Automatic Speech Recognition (ASR), Speaker Verification, Speech Synthesis, Text-to-Speech (TTS), Language Modelling, Singing Voice Synthesis (SVS), Voice Conversion (VC)

Open source, local, and self-hosted Amazon Echo/Google Home competitive Voice Assistant alternative

Transcribe any audio to text, translate and edit subtitles 100% locally with a web UI. Powered by whisper models!

A realtime voice runtime that keeps Agents talking, working, and present. Real-time Voice Runtime for AI Agents

Lingvo

Multilingual Automatic Speech Recognition with word-level timestamps and confidence

End-to-end Automatic Speech Recognition for Madarian and English in Tensorflow

🐸STT - The deep learning toolkit for Speech-to-Text. Training and deploying STT models has never been so easy.

输入主题，自动生成带配音字幕的讲解视频

用代码做短片的 Claude Code 技能，含 43 种影片风格

pytorch-kaldi is a project for developing state-of-the-art DNN/RNN hybrid speech recognition systems. The DNN part is managed by pytorch, while feature extraction, label computation, and decoding are performed with the kaldi toolkit.

🎙Speech recognition using the tensorflow deep learning framework, sequence-to-sequence neural networks

A 0.9B model for long-form transcription in 50+ languages with speaker diarization, timestamps, and acoustic event awareness

本地离线的 AI 语音与音乐创作工作室

腾讯混元开源的语音生成与编辑基础模型

本地运行的白板动画视频生成工具，上传参考音频和中文文案即可出片。

让 AI 编码代理为你生成带旁白的讲解视频

纯 CPU 实时多语言语音转文字，带字幕、说话人标签与翻译

macOS 本地离线语音听写工具，按住快捷键说话即输入

一句话让 AI 帮你完成 4K 产品宣传片的全流程制作

为编码智能体打造的本地视频工作室

输入主题，自动生成带引用的 2 分钟音频课程

本地优先的实时 2D 立绘对话客户端，自填大模型与语音

Fermion Research 开源语音识别引擎，支持多平台与多模型

自托管实时数字人系统，支持语音对话与口型同步视频

Mac 上的语音助手，按住快捷键说话即可操作电脑

基于 Flutter 的开源跨平台电子书与 PDF 阅读器

离线波斯尼亚语-英语翻译助手，支持打字、说话、拍照和朗读。

用 HTML 写画面、本地渲染带配音字幕讲解视频的 Agent Skill

基于 ggml 的 C++ 语音合成、识别与 VAD 工具

在 5 美元 ESP32-S3 上运行的开放词汇语音识别

Fish Audio 官方 Hermes Agent 语音插件

macOS 上的离线俄语语音听写工具，基于 GigaAM v3 模型

Android 课堂实时转写、点名提醒与 AI 回答助手