MangaNinjia: an automated line coloring tool to quickly color anime black and white line drawings.

General Introduction

MangaNinjia is an open source project developed by Alibaba Tongyi Visual Intelligence Lab (Ali-Vilab), focusing on the automated processing of line drawing coloring. This tool through deep learning techniques, to achieve accurate color matching of reference images, greatly improving the efficiency and quality of the creation of comics.MangaNinjia can not only automatically identify and apply the color, but also supports the user to point and click the way to fine control, so that even in the complex scene can also achieve a satisfactory color effect. The project is available on GitHub with detailed code and instructions, attracting the attention of many manga enthusiasts and professionals.

The program currently has three functions: line extraction, line coloring, and accurate line coloring.

Function List

• Automatic line coloring: Automatically identifies and applies colors based on a reference image.
• Precision Reference Following: Algorithms are used to ensure color consistency between the line drawing and the reference image.
• point-and-click coloring: Allows the user to specify the color of a specific area by clicking on a point, improving coloring accuracy.
• Multi-Reference Image Support: Color can be extracted from multiple reference images for comprehensive coloring.
• Adaptation to different line art formats: Supports a variety of line drawing input formats, such as binarized line drawing.

Using Help

Installation process

1. Cloning Warehouse:

   git clone https://github.com/ali-vilab/MangaNinjia.git
cd MangaNinjia

2. Install the dependencies:

   conda env create -f environment.yaml
conda activate MangaNinjia

3. Download the pre-trained model:
   • Download StableDiffusion from HuggingFace, clip-vit-large-patch14, controlv11psd15_lineart and Annotators model.
   • Place the downloaded model in thecheckpointsdirectory is structured as follows:
checkpoints/
├── StableDiffusion/
├── models/
├── clip-vit-large-patch14/
├── control_v11p_sd15_lineart/
└── Annotators/
├── sk_model.pth
├── MangaNinjia/
├── denoising_unet.pth
├── reference_unet.pth
├── point_net.pth
└── controlnet.pth


Usage Process

1. Run the reasoning script:

   cd scripts
bash infer.sh

• The results will be saved in theoutput/Catalog.

1. Reasoning Setup:
   • --denoise_steps: The number of denoising steps per inference is recommended to use 20-50 steps.
   • --is_lineart: This parameter is included if the input is already a line drawing and does not require additional extraction.
   • --guidance_scale_ref: Increasing the value of this parameter makes the model more inclined to be guided by the reference image.

Use of MangaNinjia

Basic Use Steps:

• Prepare the line drawing:
  • Make sure your line art image is a single-channel grayscale image with a background value of 0 and a line value close to 1. If the line art is binarized, please provide feedback in the community and we will consider further optimization.
• Upload a reference image:
  • Upload reference images that you would like the line art to be colored. These images will be used as color references.
• Point control operation (optional)
  • Click on the corresponding points on the reference image and the target line drawing to specify a color match. These points will help the system apply colors more accurately.
• Generate an image:
  • Click the "Generate" button and the system will start the coloring process according to your settings and point control information.

Detailed operating instructions:

• Image Processing: Before you start, you can click on "Process Images" to resize the image to 512x512 pixels to ensure the best performance of the model.
• Point control coloring:
  • Select the "Undo" button to undo the last tap operation.
  • If you have more than one reference image, you can use the point and click controls to select colors for different image areas for best results.
• Parameter Adjustment:
  • --denoise_steps: adjust the number of denoising steps, recommended between 20-50.
  • --is_lineart: use this parameter if the input is already a lineart and does not require additional extraction.
  • --guidance_scale_ref: Increase this value to make the model more inclined to follow the guidance of the reference image.
• Generate results:
  • The generated coloring results will be saved in the output/ directory. When checking the results, note whether the details and colors of the image are as expected, and adjust the parameters to regenerate if necessary.

MangaNinjia One-Click Installation Package

This content has been hidden by the author, please enter the verification code to view the content

Captcha:

Please pay attention to this site WeChat public number, reply "CAPTCHA, a type of challenge-response test (computing)", get the verification code. Search in WeChat for "Chief AI Sharing Circle"or"Looks-AI"or WeChat scanning the right side of the QR code can be concerned about this site WeChat public number.