VoxCPM 1.5 - Faceted Intelligence Open Source End-to-End Text-to-Speech Modeling

What is VoxCPM 1.5

VoxCPM 1.5 is an open source speech generation model released by Facade Intelligence, based on text-to-speech (TTS) technology without the need for a splitter, with several innovations and improvements. Adopting an end-to-end diffusion autoregressive architecture, it generates continuous speech waveforms directly from text, avoiding the limitations of traditional segmentation methods. The model is significantly improved in audio quality, with the sampling rate increased from 16kHz to 44.1kHz, which preserves more high-frequency details and makes speech cloning more realistic. Meanwhile, the generation efficiency is also optimized, the token rate is reduced to 6.25Hz, the computational cost is lower, and it supports real-time speech synthesis, which is suitable for real-time applications.

Features of VoxCPM 1.5

• High sample rate audio generationThe sampling rate has been increased from 16kHz to 44.1kHz, resulting in a more detailed, clearer and more natural sound, and better reproduction of tone and emotion, especially during voice cloning.
• Efficient generation of capacity: language modeling token The rate is reduced from 12.5Hz to 6.25Hz, significantly reducing computational cost while maintaining generation performance for real-time speech synthesis applications.
• zero-sample speech cloningThe speaker's tone, intonation, emotion, and other characteristics can be accurately cloned from a short reference audio clip (≥3 seconds) without additional training or registration of a speaker ID.
• Context-aware speech generation: The model understands the text content and adaptively adjusts the rhyme and style of the speech, generating a more expressive and natural flow of speech.
• Support for personalized fine-tuning: SFT and LoRA fine-tuning support is provided so that users can train personalized speech models based on their own data to meet specific needs.
• Multi-language support: Although it is primarily designed for training in English and Chinese, its architecture also provides a basis for multi-language extensions, and is expected to support more languages in the future.
• Open Source and Community SupportThe model is open-sourced on platforms such as Hugging Face, and developers are free to use, modify, and extend it, and the community provides a wealth of resources and documentation to support it.

Core Benefits of VoxCPM 1.5

• High fidelity audio generationThe 44.1kHz sampling rate produces a clearer and more detailed voice, especially in terms of timbre and emotion, which is closer to the real human voice.
• Efficient inference performance: The token generation rate is increased to 6.25Hz, the computation cost is reduced, the inference speed is faster, and the RTF (real-time factor) is as low as 0.17, which is suitable for real-time speech synthesis scenarios.
• zero-sample speech cloning: Accurate speech cloning can be achieved with only 3 seconds of reference audio, without additional training, and can quickly generate speech that is highly consistent with the reference audio.
• context-sensitive capabilityThe model can automatically adjust the rhyme and style of speech according to the content of the text, generating more expressive and natural speech, and adapting to different text scenarios.
• PersonalizationSFT (full fine-tuning) and LoRA (low-rank adaptation) fine-tuning are supported, allowing users to train personalized speech models based on their own data to meet specific needs.
• Multi-language support: English and Chinese as the core, and at the same time have a certain degree of multi-language expansion capabilities, for the future to support more languages to lay the foundation.
• Low resource dependency: No complex pre-processing or post-processing steps are required to generate speech directly from text, lowering the threshold of use and simplifying the development process.

What is the official website of VoxCPM 1.5?

• HuggingFace Model Library:: https://huggingface.co/openbmb/VoxCPM1.5

Who is VoxCPM 1.5 for?

• Speech Synthesis Developer: Developers who need efficient, high-quality speech generation capabilities for developing applications such as voice assistants, intelligent customer service, and voice broadcasting.
• content creatorVoxCPM 1.5 can be used by producers of audio podcasts and audiobooks to quickly generate high-quality voice content and increase the efficiency of their work.
• language researcher: Researchers and scholars who are interested in speech synthesis technology and wish to study areas such as speech generation and speech cloning.
• Corporate and Brand Side: Enterprises that want to enhance their brand image through personalized voice and add voice interaction features to their products or services, such as smart hardware and in-vehicle systems.
• educator: Used to create educational audio content such as online courses, language learning tools, etc. to provide a more vivid audio teaching experience.