SoniTranslate: open source video translation and dubbing solution, multi-person dubbing, adjust the speed of speech and mimic the original sound

General Introduction

SoniTranslate is a powerful and user-friendly video multilingual dubbing tool designed to provide a solution for video translation and synchronized audio. It uses advanced speech recognition and machine translation technologies to translate video content into multiple languages and keep the audio synchronized. Built on the Gradio library, the project provides a user-friendly web interface with support for multiple languages and multiple output formats for content creators and users who need multilingual video translation.

Colab notebook: https://colab.research.google.com/github/R3gm/SoniTranslate/blob/main/SoniTranslate_Colab.ipynb

Online demo: https://huggingface.co/spaces/r3gm/SoniTranslate_translate_audio_of_a_video_content

Function List

• video translation: Translate video content into multiple languages.
• synchronized audio: Keep the translated audio synchronized with the video.
• Multi-language support: Translation support for more than 80 languages.
• Subtitle Generation: Automatically generate translated subtitle files.
• speech replacement: Replaces the original speech in the video with a TTS model.
• user-friendly interface: Easy to use web interface built on the Gradio library.
• Multiple output formats: Supports output in MP3, MP4, MKV, WAV and OGG formats.

Using Help

Installation process

1. Preparing the environment: Ensure that NVIDIA drivers and CUDA 11.8.0 are installed.
2. Install FFmpeg: For processing video and audio files.
3. Install SoniTranslate::
   • Clone a GitHub repository:git clone https://github.com/R3gm/SoniTranslate.git
   • Go to the project catalog:cd SoniTranslate
   • Create and activate a Conda environment:conda create -n sonitr python=3.8(math.) genusconda activate sonitr
   • Install the dependencies:pip install -r requirements.txt
4. Running the application::
   • Launch the application:python app_rvc.py
   • Open your browser to access the local URL: http://127.0.0.1:7860

Usage Process

1. Upload Video: Upload video files to be translated in the web interface.
2. Select Language: Selection of the target language for translation.
3. Configuration options: Configure options for subtitle generation, speech replacement, etc. as needed.
4. Start translation: Click the "Start Translation" button and wait for the translation to complete.
5. Download results: Download the resulting translated video, audio or subtitle file when the translation is complete.

Detailed steps

1. Upload Video: Click the "Upload Video" button to select a local video file for uploading.
2. Select Language: Select the language to be translated in the "Target language" drop-down menu.
3. Configuration options::
   • Subtitle Generation: Check the "Generate Subtitles" option, the system will automatically generate the translated subtitle files.
   • speech replacement: Check the "Voice Replacement" option, the system will replace the original voice in the video with the TTS model.
4. Start translation: Click the "Start Translation" button and the system will start processing the video and translating it.
5. View Progress: Check the current translation progress in the "Translation Progress" column and wait for the translation to be completed.
6. Download results: Once the translation is complete, click the "Download" button, select the desired output format (MP3, MP4, MKV, WAV or OGG) and download the resulting file.

The layman's guide to getting started with AI

Help you learn how to utilize AI tools at a low cost and from a zero base.AI, like office software, is an essential skill for everyone. Mastering AI will give you an edge in your job search and half the effort in your future work and studies.

View Details>