TRELLIS: Microsoft-developed 3D asset generation model with multiple format support and flexible editing

General Introduction

TRELLIS is a large-scale 3D asset generation model developed by Microsoft. It is capable of receiving text or image cues and generating high-quality 3D assets in a variety of formats, such as radial fields, 3D Gaussians, and meshes.At the heart of TRELLIS is a unified Structured Latent Variable (SLAT) representation, which allows it to be decoded into different output formats, backed by a powerful rectifier-flow transformer designed specifically for SLAT. Pre-trained on a large 3D asset dataset containing 500,000 diverse objects, the model significantly outperforms existing methods, demonstrating flexible output format selection and native 3D editing capabilities.

Function List

• High-quality generation: Generate diverse 3D assets with complex shape and texture details.
• Versatility: Receive text or image prompts to generate a variety of 3D representations including radial fields, 3D Gaussians and meshes.
• Flexible Editing: Allows easy editing of generated 3D assets, such as generating variants of the same object or local editing of 3D assets.

Using Help

Installation process

1. precondition::
   • Recommended to run code on Linux, not tested on other platforms.
   • Conda is recommended for managing dependencies.
   • Requires Python 3.8 or higher.
   • Requires an NVIDIA GPU with 16GB or more of RAM, code has been tested on NVIDIA A100 and A6000 GPUs.
   • Requires CUDA toolkit to compile certain submodules, code has been tested on CUDA 11.8 and 12.2.
2. Installation steps::
   • Cloning repositories:

     git clone --recurse-submodules https://github.com/microsoft/TRELLIS.git
     cd TRELLIS
     

   • Install dependencies:

     . ./setup.sh --new-env --basic --xformers --flash-attn --diffoctreerast --spconv --mipgaussian --kaolin --nvdiffrast
     

Usage Process

1. Loading pre-trained models::

   from trellis.pipelines import TrellisImageTo3DPipeline
   pipeline = TrellisImageTo3DPipeline.from_pretrained("JeffreyXiang/TRELLIS-image-large")
   pipeline.cuda()
   

2. Load the image and run the pipeline::

   from PIL import Image
   image = Image.open("assets/example_image/T.png")
   outputs = pipeline.run(image, seed=1)
   

3. Render output::

   from trellis.utils import render_utils
   video = render_utils.render_video(outputs['gaussian'][0])['color']
   

4. Save results::

   import imageio
   imageio.mimsave("sample_gs.mp4", video, fps=30)