TransRouter: A Real-Time Audio Conversion Tool for Chinese-to-English Translation Based on Gemini Multimodal Modeling

General Introduction

TransRouter is a Google Gemini TransRouter is a large model real-time voice translation tool designed specifically for real-time voice translation between English and Chinese. Seamlessly integrated into video conferencing software such as Zoom, TransRouter employs streaming processing technology to ensure low latency in the translation process, and features automatic speech synthesis. The system can not only process audio signals in real time, but also automatically save the original recording and synthesized audio for subsequent review and recording. The project is completely open source, developed in Python, with good scalability and customizability.

Function List

• Realize two-way real-time voice translation between Chinese and English
• Supports seamless integration with leading conferencing software such as Zoom
• High-quality translations using Google Gemini big models
• Provides low-latency streaming audio processing
• Automatic speech synthesis output
• Supports automatic saving of original recordings and translated audio
• Provide a complete logging system
• Supports virtual audio device routing
• Configurable audio input and output parameters

Using Help

System environment preparation

1. Basic environmental requirements:
   • Python 3.8 or higher
   • macOS Operating System
   • Stable network connection
   • Google Internet company Gemini API keys
2. Audio device configuration:
   • Install the BlackHole virtual audio device:

     brew install blackhole-2ch
     

   • Configure audio devices in System Preferences:
     • Open "System Preferences > Sound".
     • Verify that the BlackHole 2ch device is properly recognized
     • Configuring the Default Input/Output Device

Installation steps

1. Cloning Project Warehouse:

   git clone https://github.com/notedit/TransRouter.git
   cd TransRouter
   

2. Create and activate a virtual environment:

   python -m venv venv
   source venv/bin/activate  # macOS
   

3. Install the dependency packages:

   pip install -r requirements.txt
   

4. Configure environment variables:
   • make a copy of .env.example file is .env
   • exist .env file to set the Gemini API key:

     GEMINI_API_KEY=your_gemini_api_key
     

Zoom Configuration Guide

1. Open the Zoom settings panel
2. Go to the Audio Settings option
3. Setting the microphone as the system default device
4. Set the speakers to "BlackHole 2ch"

Instructions for use

1. Start the program:

   python transrouter.py
   

2. The program runs automatically:
   • Initialize the audio device
   • Establishing a connection to Google Gemini
   • Starting Audio Capture and Processing
3. Document Management:
   • The recording files are saved in the recordings catalogs
   • The synthesized translated audio is saved in the synthesis catalogs
   • Runtime logs are saved in the logs catalogs
4. Operational Controls:
   • Real-time status information is displayed during program operation
   • Press Ctrl+C to stop the program safely.
   • Automatically saves all audio files when stopped

troubleshooting

1. Audio equipment issues:
   • Ensure BlackHole is properly installed and configured
   • Check the status of device identification in the system audio settings
   • Verifying Zoom's Audio Settings Configuration
2. Translation issues:
   • Confirm stable network connection
   • Verify that the API key is configured correctly
   • View log files for detailed error messages