Babelfish.ai: Browser-Run Real-Time Speech Transcription and Translation Application

General Introduction

Babelfish.ai is a real-time transcription and translation application built on Huggingface Transformer.js and Supabase Realtime. The application can load large models in the browser and run them locally to realize real-time speech-to-text and translation functions. Users can quickly transcribe voice content into text and translate it into multiple languages through simple settings and operations, which is suitable for a variety of scenarios, such as meeting recordings, cross-language communication and so on.

Function List

• Real-time speech to text: using OpenAI Whisper Realize efficient speech-to-text function.
• Real-time translation: Supports translation of transcribed text into 200 languages, using Meta's NLLB-200 model.
• Multi-Client Broadcasting: Broadcast transcribed and translated content in real-time to subscribed clients via Supabase Realtime.
• Local operation: No server support is required, all operations are done locally in the browser.
• Easy Deployment: Supports running locally and deploying to GitHub Pages.

Using Help

Installation and operation

1. local operation::
   • Cloning Warehouse:git clone https://github.com/supabase-community/babelfish.ai.git
   • Go to the project catalog:cd babelfish.ai
   • Copy the environment variable file:cp .env.local.example .env.local
   • Setting Supabase Credentials: In the .env.local Fill in your Supabase credentials in the file.
   • Install the dependencies:npm install
   • Run the development server:npm run dev
   • Open your browser to access http://localhost:3000 Available.
2. Deploy to GitHub Pages::
   • Configure your secrets in your GitHub repository settings.
   • Push the code to the main branch, GitHub Actions are automatically deployed to GitHub Pages.

Usage Functions

1. real time speech to text::
   • After opening the app, click on the "Start Transcription" button.
   • Allow the browser to access the microphone and start talking.
   • Voice content is displayed as text in real time.
2. real time translation::
   • During the transcription process, the target language is selected.
   • The transcribed text is automatically translated into the selected language and displayed on the interface.
3. Multi-Client Broadcasting::
   • Subscription clients can receive transcribed and translated content via Supabase Realtime.
   • Open the app on a different device and log in to the same Supabase account to synchronize the content you receive.

Advanced Settings

• Custom Models: can be found in the translationWorker.js is replaced with another supported translation model.
• Interface customization: Modification index.html cap (a poem) src The files in the directory can be customized with application interfaces and features.