IDProtector: a way to protect portraits from the abuse of AI-generated technology

1. Background and issues

With the rapid development of Artificial Intelligence (AI) technology, especially thediffusion modeladvances, AI has been able to generate very realistic portrait images. For example, images likeInstantIDSuch technology requires only a single photo to generate multiple new images with the same identifying features. While this technology has many application scenarios, such as creating personalized avatars, it also poses new privacy risks:

• Unauthorized abuse: Unscrupulous individuals can use this technology to generate false personal photographs for fraud, identity theft and other malicious purposes.
• privacy breach: People's photographs may be used without authorization to generate a variety of images that violate individual privacy.

Therefore, how to protect portrait photos from the misuse of this technology has become an urgent issue.

2. Shortcomings of existing methodologies

Currently, AI methods for generating portrait images fall into two main categories:

2.1 Fine-tuning based approach

• Representative Technology: DreamBooth, LoRA, etc.
• principle: By fine-tuning the AI model so that it can generate character-specific images.
• drawbacks: Multiple images are required for training, which is a complex and time-consuming process, and is not suitable for rapid generation or large-scale applications.

2.2 Encoder-based methods

• Representative Technology: InstantID, IP-Adapter, etc.
• principle: Using a pre-trained facial encoder, identity features are extracted from a single portrait photograph and then new images with the same identity features are generated.
• vantage: Only one photo is required and the process is quick and easy.
• drawbacks: Easier to abuse because no complex training process is required.

Existing protection methods are mainly aimed at fine-tuning based techniques and lack effective protection against encoder based methods.

3. IDProtector: a new protection scheme

In order to solve the above problems, this paper proposes a method namedIDProtectorof a new methodology. The core idea of the method is:

• Adding imperceptible adversarial noise to images, which prevents the AI generation model from correctly recognizing the identity of the person in the image, thus preventing the generation of images that are similar to the identity of the original image.

3.1 Main features

1. efficiency: IDProtector employs an encoder based on **ViT (Visual Transformer)** that generates adversarial noise so quickly that it takes only 0.2 seconds to protect an image, much faster than other existing methods.
2. versatility: The approach is optimized for a variety of encoder-based AI generation models, including InstantID, IP-Adapter, IP-Adapter Plus, and PhotoMaker, to provide more comprehensive protection.
3. robustness: The adversarial noise generated by IDProtector is resistant to common image processing operations such as:
   • JPEG compression: The noise remains valid even if the image is compressed.
   • Cropping and resizing: Noise can still work even if the image is cropped or resized.
   • face alignment: Resisting the effects of alignment processing of faces by AI-generated models.
4. imperceptibility: The added noise is invisible to the human visual system and does not affect the visual quality of the picture.

3.2 Principles of operation

1. Input Image Preprocessing: Resize the input portrait photo to 224 x 224 pixels and enter it into the IDProtector model.
2. Generating Adversarial Noise: The IDProtector model generates a noise map with the same dimensions as the original map and adds it to the original map.
3. Protected Image Generation: Adding noise to an image disrupts the process of extracting facial features by the AI generation model, preventing it from generating images that resemble the identity of the original image.
   
   

   

   Figure 1: Schematic of the IDProtector method. The input image is first resized to 224 × 224 pixels and then fed into the adversarial noise encoder model. The model outputs a noisy image which is added to the original image to get a protected image. This image prevents the AI generative model from correctly extracting facial features to achieve the protection effect.

4. Experimental results

4.1 Protection effects

• Reduced identity similarity: IDProtector is able to reduce the identity similarity between the generated image and the original image more significantly than existing methods. For example, on the InstantID model, IDProtector reduces the identity similarity by more than 0.4, while other methods can only reduce it by 0.1 at most.
• Facial detection rate: To ensure a comprehensive evaluation, the researchers also used the InsightFace detector to detect faces in the generated images. The results show that the IDProtector does not affect the face detection rate, which validates the reliability of its protection effect.

  

  Figure 2: Qualitative comparison with the baseline method. The IDProtector leads to more significant facial differences over the same perturbation range.

4.2 Time and image quality

• Faster: IDProtector protects an average of 0.173 seconds per image, less than 1% of the fastest existing methods.
• Higher image quality: IDProtector is less destructive to image quality and has higher PSNR and SSIM values than other methods.

  

4.3 Ability to generalize to unseen data and models

• Ability to generalize to unseen datasets: IDProtector was tested on the VGG Face dataset, which had not been seen during training, and the results show that its protection is almost unchanged between datasets, suggesting that it has a strong generalization ability.
• Ability to generalize to unseen models: In addition to the models used during training, IDProtector tested other AI-generated models, including some proprietary models (e.g., Midjourney and Jing Gou), with equally satisfactory results.

  

4.4 Robustness

• Resistance to common image processing operations: IDProtector is robust to operations such as JPEG compression, cropping, noise addition and affine transformation. Even after these processes, IDProtector is still able to protect the image effectively.

  

5. Conclusion

IDProtector provides an innovative solution to protect portrait photos from misuse by encoder-based AI generation techniques by adding imperceptible adversarial noise to the image. Its efficiency, versatility, robustness and imperceptibility make it an effective tool for protecting facial privacy.

6. Future prospects

Future research could further optimize the imperceptibility of the IDProtector while maintaining its strong protection capabilities. In addition, its application to other types of media content, such as video, could be explored to provide more comprehensive privacy protection.