MirageLSD - Decart AI Launches First Real-Time AI Video Generation Model

What is MirageLSD

MirageLSD is the world's first real-time streaming diffusion AI video model launched by Decart AI team, which is capable of generating real-time video of unlimited duration, with a latency as low as 40 milliseconds or less, and supports smooth output of 24 frames/second. Through Diffusion Forcing technology and history enhancement training, it solves the problem of error accumulation of traditional autoregressive model in long time generation and realizes unlimited video generation. Based on Hopper-optimized Mega Kernels, architecture-aware pruning and Shortcut. Distillation With MirageLSD, we have dramatically increased the generation speed while maintaining high image quality, realizing true real-time interaction.

Key features of MirageLSD

• Unlimited duration real-time video generationMirageLSD generates video streams of unlimited duration with latency as low as 40 milliseconds and supports real-time generation at 24 frames/second, which solves the problem of error accumulation in traditional video generation models over long periods of time.
• real time interactivity: Users can be prompted, converted and edited in real time during the video generation process for a continuous interactive experience.
• Low latency processing: The model achieves ultra-low latency processing of 40 milliseconds to support real-time video generation through optimization techniques such as Hopper-optimized Mega Kernels and architecture-aware pruning.

MirageLSD project address

• Technical Papers:: https://about.decart.ai/publications/mirage

Technical principles of MirageLSD

• Diffusion Forcing Technology: Frame-level generation is achieved through frame-by-frame denoising that allows the model to generate single-frame images without the full video context.
• History Enhancement Training: Introducing noisy data from historical frames during training allows the model to predict and correct errors in the input, leading to infinite generation.
• optimization strategy::
  • Hopper Optimized Mega Kernels: Optimized for the NVIDIA Hopper GPU architecture to reduce per-layer model latency.
  • Architecture-aware pruning: Reduce computation by resizing model parameters to fit the GPU architecture.
  • Shortcut Distillation: Reduce the diffusion step required for generation by training smaller models to match the denoising trajectories of larger models.

How to use

• Using the MirageLSD platform: Visit the official Mirage website provided by Decart AI: https://mirage.decart.ai/. Connect the prepared video stream to the Mirage platform.
• Preparing the Input Video Stream
  • Video Chat or Live Streaming: Use the output of a webcam or live streaming software as an input source.
  • game screen: Live feed from the game's video output.
  • computer screen: Captures the contents of the screen as input.
• Real-time conversion and editing: On the Mirage platform, users can change the content of the video stream in real time by entering text prompts or selecting preset styles. The platform supports real-time interaction, allowing users to adjust prompts or styles as needed for dynamic video transitions.
• Outputs and Applications: The converted video streams can be used directly for live streaming, gaming, video calling and other scenarios.

Modeling Advantages of MirageLSD

• Low latency with infinite generation: MirageLSD achieves ultra-low latency processing of less than 40 milliseconds and generates unlimited-length video streams in real-time at 24 frames/second. This breaks the latency and length bottlenecks of traditional video generation models, which typically generate 5-10 second clips with 10+ seconds of latency. The model's overall efficiency is improved by more than 100x through innovative CUDA Megakernel optimization and anti-drift training techniques.
• Powerful real-time interactivity: MirageLSD supports real-time dynamic response, which allows users to dynamically adjust the content during the video generation process, ensuring that the output is always consistent with the creative idea. The high degree of flexibility and control allows MirageLSD to show great potential in creative content production. Users can change the look, scene or clothing in a video in real time through simple interactions such as gesture control.

Application Scenarios for MirageLSD

MirageLSD application scenarios include: live streaming and video calling, which converts normal video calling or live streaming content in real time into user-specified scenarios, e.g., changing a realistic scene into a sci-fi world. Game development, real-time game screen can be converted into different visual styles, such as changing a normal battle scene into a lightsaber duel. Animation production and virtual dressing, providing real-time visual effect support for animation production and virtual dressing.