ACE++: a versatile open source tool for precise localized image editing with text commands

General Introduction

ACE++ is an open source project developed by the ali-vilab team at Alibaba Tongyi Lab. It is based on FLUX.1-Fill-dev model designed to enable image generation and editing through simple textual commands. The project unifies reference image generation, local editing, and controlled generation within a single framework, providing three LoRA models (portrait, object, local editing) and a more general FFT model. The code and models are free to download from GitHub and were last updated on March 11, 2025. ACE++ is suitable for developers, designers, and general users, and supports both command-line and ComfyUI Use in.

ACE++ has no significant shortcomings in face-swapping and dress-up over specialized tools, thanks to the recently launched Gemini 2.0 Flash Very bright, also the main text command control image local editing, they two effects can be compared by themselves.

Function List

• Command Generated Images: Enter a text description to generate a new image, e.g. "A girl in a police uniform".
• Local image editing: Modify specified areas of a picture, such as replacing the background or adjusting clothing.
• Reference Image Generation: Generate a new diagram of similar style based on a reference diagram.
• Multi-model selection: Three LoRA models for portrait, object, and local editing and one FFT model are available.
• ComfyUI Support: Simplified workflow through graphical interface operation.
• Customized training: Users can train models with their own data, adapted to specific tasks.
• Advanced Features: Extended applications such as super resolution, face swap, object replacement, etc. are supported.

Using Help

Installation process

ACE++ requires a configured environment to run. The following are the detailed steps:

1. Preparing the environment
   • Ensure that Python 3.8 or higher is installed on your computer, and that you can use the python --version Check.
   • To install Git, download it from the website for Windows users and run it for Mac users. brew install gitThe
   • NVIDIA GPUs with at least 12GB of RAM are recommended, CPUs can run but are slow.
2. Download Code
   • Open a terminal and run it:

     git clone https://github.com/ali-vilab/ACE_plus.git
     cd ACE_plus
     

   • This will download the project locally.
3. Installation of dependencies
   • Enter the command to install the required libraries:

     pip install -r repo_requirements.txt
     

   • For permission problems, add --user Parameters.
4. Download model
   • ACE++ relies on the FLUX.1-Fill-dev model to set environment variables:

     export FLUX_FILL_PATH="hf://black-forest-labs/FLUX.1-Fill-dev"
     

   • Download LoRA model (optional):

     export PORTRAIT_MODEL_PATH="hf://ali-vilab/ACE_Plus@portrait/comfyui_portrait_lora64.safetensors"
     export SUBJECT_MODEL_PATH="hf://ali-vilab/ACE_Plus@subject/comfyui_subject_lora16.safetensors"
     export LOCAL_MODEL_PATH="hf://ali-vilab/ACE_Plus@local_editing/comfyui_local_lora16.safetensors"
     

   • FFT model (optional):

     export ACE_PLUS_FFT_MODEL="ms://iic/ACE_Plus@ace_plus_fft.safetensors"
     

   • It can be downloaded manually from Hugging Face or ModelScope and placed in the project directory.
5. test run
   • Run the following command to check the environment:

     python infer_lora.py
     

   • If the output is normal, the installation was successful.

How to use the main features

Command Generated Images

• procedure::
  1. Enter a text description such as "an old white owl".
  2. Running:

     python infer_lora.py --prompt "1 white old owl"
     

  3. The results are saved in the output Folder.
• draw attention to sth.: The more detailed the description, the better the results. the FFT model is suitable for complex tasks.

Local image editing

• procedure::
  1. Prepare pictures (e.g. example.jpg) and mask (e.g. mask.png), masking the white area for the editing section.
  2. Running:

     python infer_lora.py --image example.jpg --mask mask.png --prompt "add a red hat"
     

  3. The edited image is automatically saved.
• finesse: The mask can be made with the drawing tool to ensure that the area is clear.

Reference Image Generation

• procedure::
  1. Prepare reference images (e.g. ref.jpg).
  2. Enter a description such as "Prairie Boy, keep the same style".
  3. Running:

     python infer_lora.py --ref ref.jpg --prompt "a boy on the grassland in the same style"
     

  4. The results retain the reference figure style.
• suggestion: Reference charts work best when the style is obvious.

Using ComfyUI

• Install ComfyUI::
  1. Download ComfyUI:

     git clone https://github.com/comfyanonymous/ComfyUI.git
     

  2. Put the ACE++ model into the ComfyUI/models Folder.
  3. Initiation:

     cd ComfyUI
     python main.py
     

• manipulate::
  1. Open your browser and visit http://localhost:8188The
  2. Import ACE++ workflows (located in the workflow/ComfyUI-ACE_Plus).
  3. Upload a picture or input text, adjust the parameters according to the interface prompts, and click Run.
• vantage: The graphical interface is suitable for novices and is more intuitive to operate.

Customized training

• Prepare data::
  1. consultation data/train.csv, prepare the training data in the format:

     edit_image,edit_mask,ref_image,target_image,prompt,data_type
     

  2. Field Description: Editorial Task Fill edit_image cap (a poem) maskReference Generation Fill ref_imageThe
• running training::
  1. configure train_config/ace_plus_lora.yaml, adjust the parameters.
  2. Running:

     python run_train.py --cfg train_config/ace_plus_lora.yaml
     

  3. The training model is saved in the examples/exp_exampleThe

Featured Function Operation

LoRA Model Switching

• methodologies::
  1. Setting environment variables, e.g. with portrait models:

     export MODEL_PATH="hf://ali-vilab/ACE_Plus@portrait/comfyui_portrait_lora64.safetensors"
     

  2. (of a computer) run infer_lora.pyThe effect is biased towards portrait optimization.
• use: Portrait models are good for faces, object models are good for items, and local editing models are good for regional adjustments.

FFT Model Use

• methodologies::
  1. Setting:

     export ACE_PLUS_FFT_MODEL="ms://iic/ACE_Plus@ace_plus_fft.safetensors"
     

  2. Running:

     python infer_fft.py
     

• specificities: Supports more tasks, but performance may not be as stable as LoRA for academic research.

Advanced Applications

• face transplant::
  1. Prepare reference face and target images.
  2. Using ComfyUI workflow ACE_Plus_FFT_workflow_referenceediting_generation.jsonInput commands such as "man facing camera".
• super-resolution::
  1. expense or outlay ACE_Plus_FFT_workflow_no_preprocess.json, enter a low-resolution diagram and description.
  2. Outputs high-definition images.

application scenario

1. Personalized avatar design
   Users enter a description or upload a photo to generate a cartoon avatar, such as "Smiley Girl with Glasses".
2. E-commerce product optimization
   Merchants edit product images, such as changing clothes to blue and adding backgrounds to boost purchase rates.
3. Movie Poster Production
   Replace actor faces with the Face Swap feature or adjust poster elements to quickly generate creative designs.

QA

1. Is ACE++ free?
   Yes, ACE++ is an open source project, and the code and models are freely available for download from GitHub.
2. What about runtime errors?
   Check the Python version, dependency installation, and model path. If you can't resolve it, submit the issue in GitHub Issues.
3. Which is better, the FFT model or the LoRA model?
   LoRA models are more stable on specific tasks, FFT models are more generalized but may have degraded performance, and are recommended to be chosen based on requirements.