OmniSVG: from text and images to generate SVG vector graphics open source project

General Introduction

OmniSVG is an open source project focused on generating high-quality vector graphics (SVG) through a multimodal model. It utilizes pre-trained visual-linguistic models to support the generation of SVG from textual descriptions or image inputs, covering a wide range of scenarios from simple icons to complex anime characters. The project provides the MMSVG dataset, including the icon and illustration subsets, with the character subset to be added in the future.OmniSVG is developed by Fudan University and the StepFun team, and the code and documentation are hosted on GitHub.The project is currently in an early stage, and the core code and models have not yet been fully released; however, the dataset and the evaluation protocol, MMSVG-Bench, have been made available, and are suitable for researchers and developers to explore.

Function List

• Supports SVG generation from text descriptions to output editable vector graphics.
• Image to SVG support to convert raster images to vector format.
• Provides role consistency generation to maintain style uniformity based on reference.
• Contains the MMSVG dataset, providing resources for icons and illustrations.
• Provides the MMSVG-Bench protocol to evaluate the quality of SVG generation.
• Supports complex SVG output for highly detailed illustrations and character designs.
• The core code will be open-sourced in the future, allowing users to modify and optimize the model.

Using Help

Project preparation

OmniSVG does not currently provide full deployment files, and the core code and pre-trained models have not yet been released. The following preparatory steps, based on the available documentation, are provided for users to access the data and understand the project:

1. Clone Code Repository
   Open a terminal and run the following command to get the OmniSVG repository:

   git clone https://github.com/OmniSVG/OmniSVG.git
   

   Go to the project catalog:

   cd OmniSVG
   

   The current repository contains the following files:

   • assets/: Stores demo images and GIFs to show the generated results.
   • data/: Contains example data or metadata for specific purposes. README.mdThe
   • LICENSE: The project is under an open source license (see documentation for specific terms).
   • README.md: Provides a brief description of the project, links to datasets, and future plans.
2. Checking the Python Environment
   OmniSVG relies on Python 3.8+ and machine learning libraries (e.g. PyTorch, Transformers). While it is not yet possible to provide requirements.txtIt is recommended that the following base libraries be installed for subsequent code releases:

   pip install torch transformers pillow numpy
   

   If the code is released in the future, additional libraries may be required; refer to the official documentation at that time.

3. Download MMSVG Dataset
   The MMSVG dataset is the project's current core resource and contains two subsets hosted at Hugging Face:
   • MMSVG-Icon:https://huggingface.co/OmniSVG/MMSVG-Icon
   • MMSVG-Illustration:https://huggingface.co/OmniSVG/MMSVG-Illustration
     Operational Steps:
   • Visit the link above to download the dataset.
   • Unzip the file, it is recommended to store it in the OmniSVG/datasets/ directory (can be created manually).
   • The dataset contains about 2 million SVG data covering icons and illustrations suitable for training or analysis.
4. View Demo Resources
   assets/ folder contains demo GIFs (e.g. omnisvg-teaser.gif), showing the process of text to SVG and anime character generation. Users can view it in the following ways:
   • show (a ticket) assets/ directory, load the GIF with your browser or image viewer.
   • Refer to the input and output examples in the GIF for modeling capabilities (e.g., generating anime characters with complex paths).
5. Watch for official updates
   The current code and model are not released, so we recommend following the channels below for updates:
   • GitHub repositories:https://github.com/OmniSVG/OmniSVG
   • The official website of the project:https://omnisvg.github.io
   • Hugging Face page:https://huggingface.co/OmniSVG
     There are official plans to release a subset of MMSVG-Character and the full code, which is expected to include deployment scripts and model weights.

Data set use

The MMSVG dataset is the main resource currently available, and the following details how to use it:

1. MMSVG-Icon subset

• useIncludes minimalist SVG icons for UI design, web development, and modeling.
• procedure::
  • After downloading the dataset, unzip it to datasets/MMSVG-Icon/The
  • Use Python scripts to navigate through the data, for example:

    import os
    for file in os.listdir("datasets/MMSVG-Icon"):
    if file.endswith(".svg"):
    print(file)
    

  • Open the SVG file (using a browser or Inkscape) and view the icon style.
• appliance: Analyze icon structures, extract path data, or pre-train for generating models.

2. MMSVG-Illustration subset

• use: Contains complex illustrations suitable for artistic design and multimodal studies.
• procedure::
  • Unzip to datasets/MMSVG-Illustration/The
  • Use a similar script to traverse the illustration file and examine the SVG content.
  • Illustrations can be imported into design software (e.g. Adobe Illustrator) for editing.
• specificities: Illustrations contain multiple paths and colors, demonstrating the model's ability to generate complex shapes.

3. MMSVG-Bench assessment

• use: MMSVG-Bench is a standardized protocol for evaluating the quality of SVG generation.
• current state: The evaluation script has not yet been released, pending code updates.
• preliminary: Familiarize yourself with the structure of the dataset and record the characteristics of the icons and illustrations (e.g., number of paths, color distribution) in preparation for future evaluations.
• Intended function: According to the official description, MMSVG-Bench will provide metrics (e.g., path accuracy, style consistency) to help users test models.

Function Preview

While the core code has not been released, the official documentation and demos provide detailed descriptions of the following features, which users can learn about in advance:

1. Text Generation SVG

• functionality: The user enters a text description (e.g. "a red pentagram") and the model generates an SVG file.
• demonstrations::assets/omnisvg-teaser.gif Demonstrate the generation process from text to icons.
• Intended operation: After future code releases, users can run scripts, enter prompt words, and output vector graphics.
• specificities: Generate results that are scalable and lossless for web and print design.

2. Image to SVG

• functionality: Converts PNG or JPG images to editable SVG.
• demonstrations: Official page showing examples of raster images converted to vector paths.
• use: Supports complex image conversion, preserving outlines and colors for UI icon optimization.
• Anticipated process: Upload the image, run the conversion script, and get the SVG output.

3. Role consistency generation

• functionality: Generate new character graphics with a consistent style based on a reference SVG or image.
• demonstrations::assets/OmniSVG-demo-gen-proc-anime-1080.gif Demonstrate anime character generation.
• dominance: Supports high detail generation that maintains character features (e.g., colors, silhouettes).
• appliance: Suitable for game design, generating multi-action characters.

caveat

• Project Status: Currently, only datasets and demo resources are available, and core functionality needs to wait for code release.
• Hardware Recommendations: Running models in the future may require a GPU (e.g., NVIDIA graphics card) to accelerate generation.
• documentation reference::README.md and the official website (https://omnisvg.github.io) to provide up-to-date information.
• Community involvement: Get developer support by asking questions or providing feedback via GitHub Issues.

application scenario

1. Icon data set analysis
   Designers use the MMSVG-Icon subset to study the path structure of minimalist icons for UI design inspiration.
2. Illustration resource development
   Artists browse the MMSVG-Illustration subset to extract complex illustrations optimized for advertising or publication.
3. Modeling Studies
   Using the MMSVG dataset, the researchers developed new SVG generation models to explore multimodal techniques.
4. Educational resources
   Teachers explain the principles of vector graphics generation by demonstrating GIFs to help students understand the use of AI in design.

QA

1. Is OmniSVG currently working?
   Currently, only the dataset and demo resources are available, the core code and model have not been released yet, we need to wait for the official update.
2. How is the MMSVG dataset used?
   Download a subset of icons or illustrations, unzip and view with Python script or design software, suitable for training or design.
3. Programming experience required?
   No programming is required to browse the dataset; basic Python knowledge may be required to run the model in the future, and the documentation will provide guidance.
4. Are the datasets free?
   Yes, MMSVG-Icon and MMSVG-Illustration are free and open at Hugging Face.
5. How to generate anime characters?
   Currently waiting for the Role Conformance Generation code to be released, the demo GIF shows the expected result.