SHMT: Self-Supervised Hierarchical Makeup Transfer Model, Virtual Makeup, Migrating Makeup to New Portraits

General Introduction

SHMT (Self-supervised Hierarchical Makeup Transfer) is a self-supervised hierarchical make-up transfer project based on potential diffusion modeling, aiming to achieve high quality transfer of make-up effects through unsupervised learning methods. The project adopts the "decoupling and reconstruction" paradigm, which eliminates the misleading effects of inaccurate pseudo-pairing data. In order to adapt to diverse makeup styles, the hierarchical texture details are decomposed by Laplace pyramid and selectively introduced into the content representation.The research results of the SHMT project have been published in NeurIPS 2024, demonstrating excellent results in handling both simple and complex makeup styles.

Function List

• Self-supervised learning: makeup migration without paired data.
• Layered texture details: decompose and reconstruct texture details using Laplace pyramids.
• Efficient Migration: Maintains efficiency and high quality when working with multiple makeup styles.
• Diverse applications: suitable for image processing, style migration, computer vision and many other fields.

Using Help

Installation process

1. Clone the SHMT project repository:

   git clone https://github.com/Snowfallingplum/SHMT.git
   cd SHMT
   

2. Create and activate the virtual environment:
```bash
python -m venv venv
source venv/bin/activate # on Windows use `venv\Scripts\activate`

3. Install project dependencies:

   pip install -r requirements.txt
   

4. (Optional) Install CUDA to support GPU acceleration.

Tutorials

1. Running the main program

The main program is located atmain.pyfile, run the following command to start it:

python main.py --input your_image_path --style your_desired_style

In the above command, the--inputparameter specifies the input image path.--styleparameter specifies the target makeup style.

2. Detailed adjustments

existconfig.yamlfile, you can adjust model parameters and detail settings. For example, the number of layers of the Laplace pyramid and the intensity of the reconstruction details can be adjusted.

3. Output of results

After running the main program, the results will be saved in the specified output folder. You can open and view the makeup migration results with an image viewing tool.

typical example

Below is a simple sample code:

from shmt import SHMT
# Initialize the model
model = SHMT()
# load input image and target style
input_image = 'path/to/input/image.jpg'
style = 'glamorous'
# Perform a make-up migration
output_image = model.transfer(input_image, style)
# Save the results
output_image.save('path/to/output/image.jpg')

common problems

1. How to deal with poor quality of input images?
   Ensure that the input image is clear and high resolution. Low-quality images may affect makeup migration results.
2. How do you adjust the intensity of your makeup style?
   In the configuration fileconfig.yamlAdjust the relevant parameters in, for examplestyle_strengthThe