OPR: the AI tool that intelligently restores yellowed, worn-out old photos

General Introduction

"Bringing Old Photos Back to Life is an open-source project developed by a team of Microsoft researchers that focuses on restoring old photos using AI technology. It is based on a deep learning approach that can deal with serious degradation issues in photos, such as scratches, blurring and fading, to bring historical images back to life. The project was presented as an oral presentation at CVPR 2020 (Conference on Computer Vision and Pattern Recognition) and received widespread attention. Users can access the code and pre-trained models via GitHub for research or personal photo restoration needs. The project not only provides technical implementations, but also includes detailed documentation and examples for developers and hobbyists.

Experience it online at https://colab.research.google.com/drive/1NEm6AsybIiC5TwTU_4DqDkQO0nFRB-uA?usp=sharing

Function List

• Overall quality of old photographs improved: Improve the clarity, contrast and color of your photos with AI algorithms.
• Scratch Auto Repair: Detects and repairs physical scratches on photos, restoring original image details.
• Face Area Enhancement: Fine-tuning for face areas to enhance the clarity of facial features.
• High Resolution Support: Supports processing of higher resolution photos for professional restoration needs.
• user interface operation: Provides a GUI interface to simplify the photo upload and restoration process.
• Customized Training Models: Allows users to train specific datasets on demand, extending the applicable scenarios.

Using Help

Installation process

To use "Bringing Old Photos Back to Life", you need to complete the environment configuration and code installation. Below are the detailed steps for Ubuntu system (Windows users need to adjust according to the environment):

1. Preparing the environment

• system requirements: Ubuntu (recommended) with Nvidia GPU and CUDA support.
• Python version: Make sure Python 3.6 or above is installed.
• Dependent Installation::
  1. Open a terminal and enter the following command to install the necessary libraries:

     pip install torch torchvision numpy opencv-python PySimpleGUI
     

  2. If GPU support is required, ensure that CUDA and the corresponding version of PyTorch are installed.

2. Downloading code and models

• clone warehouse::

  git clone https://github.com/microsoft/Bringing-Old-Photos-Back-to-Life.git
  cd Bringing-Old-Photos-Back-to-Life

• Synchronized batch standardization module::
  1. go intoFace_Enhancement/models/networks/Catalog:

     cd Face_Enhancement/models/networks/
     git clone https://github.com/vacancy/Synchronized-BatchNorm-PyTorch
     cp -rf Synchronized-BatchNorm-PyTorch/sync_batchnorm ./
     cd ../../../
     

  2. go intoGlobal/detection_models/Repeat for the catalog:

     cd Global/detection_models/
     git clone https://github.com/vacancy/Synchronized-BatchNorm-PyTorch
     cp -rf Synchronized-BatchNorm-PyTorch/sync_batchnorm ./
     cd ../../
     

• Download pre-trained model::
  1. go intoFace_Enhancement/Catalog, Download Face Enhancement Models:

     cd Face_Enhancement/
     wget https://github.com/microsoft/Bringing-Old-Photos-Back-to-Life/releases/download/v1.0/face_checkpoints.zip
     unzip face_checkpoints.zip
     cd ../
     

  2. go intoGlobal/Catalog, download Global Repair Model:

     cd Global/
     wget https://github.com/microsoft/Bringing-Old-Photos-Back-to-Life/releases/download/v1.0/global_checkpoints.zip
     unzip global_checkpoints.zip
     cd ../
     

• Face Detection Model::
  1. go intoFace_Detection/directory, download the test file:

     cd Face_Detection/
     wget http://dlib.net/files/shape_predictor_68_face_landmarks.dat.bz2
     bzip2 -d shape_predictor_68_face_landmarks.dat.bz2
     cd ../
     

3. Test installation

• Run the sample command to confirm that the environment is correct:

  python run.py --input_folder ./test_images --output_folder ./output --GPU 0
  

• If successful, the restored photo will be saved in the./output/final_output/Catalog.

Usage

Operating from the command line

1. Preparing the input folder: Place the old photos to be restored in a folder (e.g.test_images/).
2. Run the repair command::
   • No scratch repair:

     python run.py --input_folder /path/to/test_images --output_folder /path/to/output --GPU 0
     

   • Contains scratch repair:

     python run.py --input_folder /path/to/test_images --output_folder /path/to/output --GPU 0 --with_scratch
     

3. View Results: The restored photos are automatically saved to the specified output folder (e.g./path/to/output/final_output/).

Operation via GUI interface

1. Starting the GUI::

   python GUI.py
   

2. procedure::
   • Click on the "Select Photos" button to upload the old photos to be restored.
   • Click the "Modify Photo" button and wait for the process to complete (time depends on the size of the photo and hardware performance).
   • When processing is complete, the results are displayed on the interface and also saved to the./output/Folder.
   • Click "Exit Window" to close the program.

Featured Functions Operation Procedure

1. Scratch repair

• Applicable Scenarios: There are obvious physical scratches in the photos.
• Operating Methods::
  1. On the command line, add--with_scratchparameter, or check the relevant option in the GUI (if supported).
  2. The system will automatically detect the scratched area and repair it, the process may take a little longer.
  3. Check the output, the scratched portion should be filled smoothly.

2. Face enhancement

• Applicable Scenarios: Blurred faces or missing details in photos.
• Operating Methods::
  1. assureFace_Enhancement/The models in the directory are loaded correctly.
  2. When running, the system will automatically recognize the face area and refine it.
  3. Face details (such as eyes and corners of the mouth) will be clearer in the output photo.

3. High-resolution processing

• Applicable Scenarios: Need to fix high resolution scanned photos.
• Operating Methods::
  1. On the command line, add--HRParameters:

     python run.py --input_folder /path/to/test_images --output_folder /path/to/output --GPU 0 --HR
     

  2. Note: Since the model is trained at 256x256 resolution by default, higher resolutions may require more computational resources.

caveat

• pathway issue: It is recommended to use absolute paths (e.g./home/user/test_images) to avoid mistakes.
• performance optimization: The project has not been optimized for operating speed, and large photos may be slow to process.
• Resolution Limit: Best results at 256x256 pixels, oversized resolutions may require chunking.

With the above steps, users can easily get started restoring old photos, whether for personal memories or research purposes, and get satisfactory results.