Faces2Anime

Cartoon Style Transfer in Faces using GANs

./imgs/tx_interpolation.gif ./imgs/tx_interpolation.PNG This project is aim to accomplish style transfer from human faces to anime / manga / cartoon styles.

Results

Trained styles

The styles used below are from our training dataset.

./imgs/trained_style.PNG

./imgs/trained_style2.PNG

Un-trained styles

The styles used below are randomly collected from the internet.

./imgs/un-trained_style.PNG

./imgs/un-trained_style2.PNG

Mean style transference

Mean style denotes the mean of all texture codes of our training styles.

./imgs/mean_style_transfer.PNG

Texture interpolation

Tx rate = n
tx1 : tx2 = 1-n : n

./imgs/tx_interpolation2.PNG

Structure interpolation

Unfortunately, our model fail to interpolate structures as it can only be applied to texture semantic style transfer.

Stu rate = n
stu1 : stu2 = 1-n : n

./imgs/stu_interpolation.PNG

Results with removing backgrounds using removebg.

./imgs/stu_interpolation2.PNG

Comparison

Compared methods
- U-GAT-IT [paper] [web] [github]
- Artbreeder [web]
- neural-style-tf [github]
- deepart io [web]
- Deep Image Analogy [paper] [github]
- FaceBlit [paper] [github]
- Adobe Characterizer [web] [video]
- original SwapAE [paper] [github]
- WCT2 [paper] [github]

./imgs/comparison1.PNG

./imgs/comparison2.PNG

Lighting preservation

./imgs/comparison3.PNG

Further Transformation

In-Domain GAN Inversion for Real Image Editing

Zhu et al. 2020. [paper] [github]

./imgs/further_in-domain.PNG

Toonify!

Justin Pinkney. 2020. [web]

./imgs/further_toonify.PNG

Datasets

:warning: Warning: All images are only used for research purpose. Prohibited for commercial use.

Training data

Anime
- Images randomly collected from WEBTOON. (Total: 22,741; Titles: 128)
- Images generated from StyleGAN2 anime pre-train model. (Total: 300)
Human faces
- Images generated from StyleGAN2 FFHQ pre-train model. (Total: 3,802)
- Celebrity faces selected from the CelebA dataset and randomly collected from the internet. (Total: 1,311)

Testing data

Anime
- Images randomly collected from the internet. (Total: 170)
Human faces
- Images randomly collected from the internet. (Total: 17)
- Images selected from the FFHQ dataset. (Total: 63)

Acknowledgement

The related research papers:
- Swapping Autoencoder for Deep Image Manipulation [paper] [github]
- Learning to Cartoonize Using White-box Cartoon Representations [paper] [github]
Face cartoonization for pre-processing here.
The idea of this project is highly inspired from naver-webtoon-faces by bryandlee.
Our code implementation is based on swapping-autoencoder-pytorch by rosinality.

BibTeX Citation

If you find this work useful for your research, please cite:

< English version >

@mastersthesis{huang2021cartoon,
  author={Cheryl Huang},
  title={Cartoon Style Transfer in Faces using Generative Adversarial Networks},
  school={National Taiwan University of Science and Technology},
  year={2021},
  note={\url{https://hdl.handle.net/11296/j8jm9n}}
}

< Chinese version >

@mastersthesis{huang2021cartoon,
  author={黃竹萱},
  title={基於生成對抗網路之臉部卡通風格轉換},
  school={國立臺灣科技大學資訊工程研究所},
  year={2021},
  note={\url{https://hdl.handle.net/11296/j8jm9n}}
}