AI Personal Learning
and practical guidance
讯飞绘镜

SVFR: A Unified Framework for Realizing Video Face Repair, Repairing Black and White, Blurry Portrait Old Videos

General Introduction

SVFR (Stable Video Face Restoration) is a unified framework for video face restoration that supports Basic Face Restoration (BFR), colorization, repair, and their combination tasks. The framework utilizes generative and motion priors to integrate task-specific information through a unified face restoration framework to improve restoration quality and temporal stability.SVFR is designed to address the challenges of temporal consistency, motion artifacts, and limited high-quality video data in video face restoration. The project was developed by Zhiyao Wang et al. and is open-sourced on GitHub, where the code and pre-trained models are available for academic and non-commercial research.

SVFR:实现视频人脸修复的统一框架,修复黑白、模糊的人像老视频-1


 

SVFR:实现视频人脸修复的统一框架,修复黑白、模糊的人像视频-1

 

Function List

  • Basic Face Restoration (BFR): Fixes face details in low-quality videos.
  • coloring: Add color to black and white video.
  • fixes: Fix the missing parts of the video.
  • Portfolio of tasks: Supports any combination of BFR, coloring, and restoration.
  • time consistency: Enhancing the temporal consistency of video restoration through generation and motion prior.
  • Pre-trained models: Provide a variety of pre-trained models to facilitate users to get started quickly.

 

Using Help

Installation process

  1. Creating a Conda Environment::
   conda create -n svfr python=3.9 -y
conda activate svfr
  1. Installing PyTorch: Choose the right CUDA version for your hardware, for example:
   pip install torch==2.2.2 torchvision==0.17.2 torchaudio==2.2.2
  1. Installation of dependencies::
   pip install -r requirements.txt
  1. Download Checkpoints: Manually download and place in the specified directory.

Usage Process

  1. single-task reasoning::
   python3 infer.py --config config/infer.yaml --task_ids 0 --input_path ./assert/lq/lq1.mp4 --output_dir ./results/
  • task_id: 0 Denotes Basic Face Restoration (BFR)
  • task_id: 1 Indicates coloring
  • task_id: 2 Indicates restoration
  1. multitask combinatorial reasoning::
   python3 infer.py --config config/infer.yaml --task_ids 0,1,2 --input_path ./assert/lq/lq3.mp4 --output_dir ./results/
  • task_ids: 0,1 Indicates BFR and coloring
  • task_ids: 0,1,2 Indicates BFR, coloring and restoration
  1. Reasoning with repair masks::
   python3 infer.py --config config/infer.yaml --task_ids 0,1,2 --input_path ./assert/lq/lq3.mp4 --output_dir ./results/ --mask_path ./assert/mask/lq3.png

Detailed function operation flow

  1. Basic Face Restoration (BFR)::
    • Select the input video path and output directory.
    • Set the task ID to 0.
    • Run the reasoning command to generate the repaired video.
  2. coloring::
    • Select the input video path and output directory.
    • Set the task ID to 1.
    • Run the inference command to generate the colorized video.
  3. fixes::
    • Select the input video path and output directory.
    • Set the task ID to 2.
    • Run the reasoning command to generate the repaired video.
  4. Portfolio of tasks::
    • Select the input video path and output directory.
    • Set the task ID to the ID of the combined task, e.g. 0,1 for BFR and coloring.
    • Run the inference command to generate the video after the combined task processing.
  5. Reasoning with repair masks::
    • Select the input video path, output directory and mask file path.
    • Set the task ID to the ID of the combined task, e.g. 0,1,2 for BFR, Coloring and Repair.
    • Run the inference command to generate the video with the repair mask processed.
May not be reproduced without permission:Chief AI Sharing Circle " SVFR: A Unified Framework for Realizing Video Face Repair, Repairing Black and White, Blurry Portrait Old Videos
en_USEnglish