Multimodal real-time interactive products

Total 27 articles posts
RealtimeVoiceChat:低延迟与AI进行自然口语对话

RealtimeVoiceChat: low-latency natural spoken conversation with AI

General Introduction RealtimeVoiceChat is an open source project focused on real-time, natural conversations with artificial intelligence via voice. Users use a microphone to input their voice, and the system captures the audio through a browser, quickly converts it to text, and a large-scale language model (LLM) generates back...
3mos ago
0798
VideoMind:视频按时间戳定位内容与问答的开源项目

VideoMind: video by timestamp positioning content and Q&A open source project

General Introduction VideoMind is an open source multimodal AI tool focused on inference, Q&A and summary generation for long videos. It was developed by Ye Liu of the Hong Kong Polytechnic University and a team from Show Lab at the National University of Singapore. The tool mimics human understanding of video...
2mos ago
01.1K
Step-Audio:多模态语音交互框架,识别语音并使用克隆语音交流等功能

Step-Audio: a multimodal voice interaction framework that recognizes speech and communicates using cloned speech, among other features

Comprehensive Introduction Step-Audio is an open source intelligent speech interaction framework designed to provide out-of-the-box speech understanding and generation capabilities for production environments. The framework supports multi-language dialog (e.g., Chinese, English, Japanese), emotional speech (e.g., happy, sad), regional dialects (e.g., Cantonese, Szechuan ...
6mos ago
02K
SpeechGPT 2.0-preview:实时交互的端到端拟人语音对话大模型

SpeechGPT 2.0-preview: an end-to-end anthropomorphic speech dialog grand model for real-time interaction

SpeechGPT 2.0-preview is the first anthropomorphic real-time interaction system introduced by OpenMOSS, which is trained based on millions of hours of speech data. The system is equipped with anthropomorphic spoken expression and 100ms low latency response, supporting natural and smooth real...
6mos ago
01.7K
百聆 (Bailing):低延时的开源语音对话助手,轻松实现自然对话交流

Bailing: a low-latency open source voice dialog assistant that easily realizes natural conversational exchanges

Comprehensive Introduction Bailing (Bailing) is an open source voice conversation assistant designed to engage in natural conversations with users through speech. The project combines speech recognition (ASR), voice activity detection (VAD), large language modeling (LLM) and speech synthesis (TTS) technologies to achieve...
7mos ago
01.9K
OmAgent:构建多模态智能设备的智能体框架

OmAgent: an intelligent body framework for building multimodal smart devices

Comprehensive Introduction OmAgent is a multimodal intelligent body framework developed by Om AI Lab, aiming to provide powerful AI-powered features for smart devices. By integrating state-of-the-art multimodal base models and intelligent body algorithms, the project enables developers to create efficient smart devices on a variety of...
7mos ago
01.7K
小智 AI 聊天机器人:打造你的AI聊天伴侣,轻松实现语音对话和智能互动

Xiaozhi AI Chatbot: Build your AI chatting companion, easily realize voice conversation and intelligent interaction

Comprehensive Introduction Xiaozhi AI Chatbot is an open source project based on the ESP32 development board, designed to help users build their own AI chat companion. The project was developed by Shrimp and is mainly used for teaching purposes to help more people get started with AI hardware development and to understand how to apply large language models to real...
5mos ago
02.4K
VITA:开源视觉与语音实时交互的多模态大语言模型

VITA: Open Source Multimodal Large Language Model for Real-Time Interaction between Vision and Speech

General Introduction VITA is a leading open source interactive multimodal large language modeling project, pioneering the ability to achieve true full multimodal interaction. The project launched VITA-1.0 in August 2024, pioneering the first open source interactive fully-modal large language model.2024...
7mos ago
01.6K
TransRouter:基于Gemini多模态模型,实时中英互译的音频转换工具

TransRouter: A Real-Time Audio Conversion Tool for Chinese-to-English Translation Based on Gemini Multimodal Modeling

TransRouter is a real-time voice translation tool based on Google's Gemini model, specifically designed for real-time voice translation between English and Chinese. The tool can be seamlessly integrated into video conferencing software such as Zoom, providing an easy way for cross-language...
7mos ago
01.6K
Fish Agent:端到端AI语音克隆助手,实时语音对话助理,Fish Speech衍生项目

Fish Agent: end-to-end AI voice cloning assistant, real-time voice conversation assistant, Fish Speech spin-off project

Comprehensive Introduction Fish Speech Derivative Project Fish Agent is a revolutionary end-to-end AI speech cloning system developed based on the V0.1 3B model architecture. As a fully end-to-end speech clone processing system, its most important feature is the use of innovative speechless...
7mos ago
02K
Megrez-3B-Omni:端侧多模态理解模型,支持文本、图像、音频多模态理解和分析

Megrez-3B-Omni: an end-side multimodal understanding model supporting text, image, and audio multimodal understanding and analysis

Comprehensive Introduction Infini-Megrez is an edge intelligence solution developed by the unquestioned core dome (Infinigence AI), aiming to achieve efficient multimodal understanding and analysis through hardware and software co-design. At the core of the project is the Megrez-3B model, which supports graph...
7mos ago
01.5K