Code Monkey home page Code Monkey logo

cartoongan's Introduction

CartoonGAN

Torch implementation for Yang Chen, Yu-Kun Lai, Yong-Jin Liu. CartoonGAN: Generative Adversarial Networks for Photo Cartoonization, CVPR2018

This code borrows from early version of CycleGAN.

Installation

The same as CycleGAN

luarocks install nngraph
luarocks install class
luarocks install https://raw.githubusercontent.com/szym/display/master/display-scm-0.rockspec
  • Clone this repo:
git clone https://github.com/FlyingGoblin/CartoonGAN.git
cd CartoonGAN

Apply pre-trained Model

  • Download the pre-trained model:
bash ./pretrained_models/download_model.sh
  • Generate Shinkai style images:
DATA_ROOT=./datasets/<your test data> name=Shinkai model=one_direction_test phase=test loadSize=256 fineSize=256 resize_or_crop="scale_width" th test.lua

Dataset

  • For the photo: All the images in our paper are from CycleGAN: https://github.com/junyanz/pytorch-CycleGAN-and-pix2pix (XXX2photo).
  • For the Cartoon image: Because all the cartoon images are from films, I don't know if I have the right to upload them. I don't know if there will be any copyright problem, I am sorry.
    • I just calculate the SSIM and PSNR between frames to remove too samiliar images.
    • The cartoon dataset should not be too small.

Train

  • Dataset: Prepare your datasets in ./datasets/, it should have subfolders train_A, train_B, train_B_edge
  • Train a model with init process:
DATA_ROOT=./datasets/<your data> name=<your name> th train.lua
  • Train a model with an init model:
DATA_ROOT=./datasets/<your data> name=<your name> init_model=<path to your init model> th train.lua

Test

  • Dataset: Prepare your datasets in ./datasets/, it should have subfolders test_A
DATA_ROOT=./datasets/<your test data> name=<your name> model=one_direction_test phase=test loadSize=256 fineSize=256 resize_or_crop="scale_width" th test.lua

BlaBla

  • My email: [email protected] can NOT send emails because of my graduation, and it won't be able to receive email very soon (maybe I have already missed some emails). So please contact me by [email protected].
  • In fact I didn't mean to clean up the code before because of laziness _(:з」∠)_, but after receiving several contact asking me about the reproduction, I decided to clean them up to avoid "rebuilding the wheel".
  • I hope this code can speed up your work ~
  • 发现我这个作者的代码的星星在GitHub里CartoonGAN相关项目里才排第5啊_(:з」∠)_,而且和前面的差好远_(:з」∠)_我唯一一个有可能有星星的项目啊_(:з」∠)_o(╥﹏╥)o

cartoongan's People

Contributors

flyinggoblin avatar snowonion avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

cartoongan's Issues

关于反卷积层

Hello,作者您好,我注意到你设计的网络中,每个反卷积层后都加了一层卷积层,这个卷积层是为了消除棋盘效应而设计的吗?

关于预训练模型的问题

你好,我们根据您的论文在实验的时候尝试着,先训练G,然后在将其作为预训练的模型并载入,继续训练,此时训练的是G和D两个网络,但是实验结果和您论文无初始化训练的结果相似。因此,重新思考预训练的方式。提出一个方法,思路如下:在100个epoch中取出10个epoch,仅训练G,然后在此基础上,在剩余的90个epoch中,训练G和D两个网络。请问是否能达到与您论文中的预训练同样的效果?
另外,还有一个疑问,在预训练的阶段是否必须进行大样本的训练?
谢谢!

关于损失值以及预训练次数的问题

各位大佬好!预训练次数是多少啊,以及,我如何通过生成器损失值,判别器损失值和内容损失值,这三个属性判断我的模型训练争取,我的数据如下所示,不胜感激!
Pre-training start!
[1/50] - time: 340.47, Recon loss: 20.638
[2/50] - time: 337.64, Recon loss: 16.235
[3/50] - time: 337.67, Recon loss: 14.199
[4/50] - time: 337.59, Recon loss: 12.793
[5/50] - time: 337.73, Recon loss: 11.443
[6/50] - time: 337.51, Recon loss: 10.535
[7/50] - time: 337.65, Recon loss: 9.870
[8/50] - time: 337.47, Recon loss: 9.298
[9/50] - time: 337.51, Recon loss: 8.800
[10/50] - time: 337.45, Recon loss: 8.343
[11/50] - time: 337.39, Recon loss: 7.938
[12/50] - time: 337.38, Recon loss: 7.583
[13/50] - time: 337.44, Recon loss: 7.277
[14/50] - time: 337.50, Recon loss: 7.022
[15/50] - time: 337.41, Recon loss: 6.802
[16/50] - time: 337.40, Recon loss: 6.611
[17/50] - time: 337.34, Recon loss: 6.434
[18/50] - time: 337.34, Recon loss: 6.276
[19/50] - time: 337.43, Recon loss: 6.126
[20/50] - time: 337.45, Recon loss: 5.982
[21/50] - time: 337.38, Recon loss: 5.848
[22/50] - time: 337.43, Recon loss: 5.721
[23/50] - time: 337.36, Recon loss: 5.589
[24/50] - time: 337.33, Recon loss: 5.473
[25/50] - time: 337.36, Recon loss: 5.350
[26/50] - time: 337.31, Recon loss: 5.234
[27/50] - time: 337.24, Recon loss: 5.125
[28/50] - time: 337.27, Recon loss: 5.020
[29/50] - time: 337.33, Recon loss: 4.916
[30/50] - time: 337.32, Recon loss: 4.819
[31/50] - time: 337.47, Recon loss: 4.729
[32/50] - time: 337.37, Recon loss: 4.636
[33/50] - time: 337.24, Recon loss: 4.555
[34/50] - time: 337.32, Recon loss: 4.474
[35/50] - time: 337.43, Recon loss: 4.407
[36/50] - time: 337.32, Recon loss: 4.330
[37/50] - time: 337.32, Recon loss: 4.262
[38/50] - time: 337.36, Recon loss: 4.200
[39/50] - time: 337.38, Recon loss: 4.130
[40/50] - time: 337.37, Recon loss: 4.072
[41/50] - time: 337.38, Recon loss: 4.019
[42/50] - time: 337.37, Recon loss: 3.960
[43/50] - time: 337.38, Recon loss: 3.909
[44/50] - time: 337.37, Recon loss: 3.857
[45/50] - time: 337.44, Recon loss: 3.809
[46/50] - time: 337.40, Recon loss: 3.763
[47/50] - time: 337.45, Recon loss: 3.721
[48/50] - time: 337.43, Recon loss: 3.671
[49/50] - time: 337.46, Recon loss: 3.642
[50/50] - time: 337.46, Recon loss: 3.592
training start!
[1/100] - time: 139.16, Disc loss: 1.919, Gen loss: 1.084, Con loss: 3.581
[2/100] - time: 139.10, Disc loss: 1.827, Gen loss: 1.275, Con loss: 3.535
[3/100] - time: 139.21, Disc loss: 1.812, Gen loss: 1.300, Con loss: 3.621
[4/100] - time: 139.14, Disc loss: 1.768, Gen loss: 1.352, Con loss: 3.628
[5/100] - time: 139.15, Disc loss: 1.699, Gen loss: 1.387, Con loss: 3.637
[6/100] - time: 139.18, Disc loss: 1.621, Gen loss: 1.363, Con loss: 3.697
[7/100] - time: 138.93, Disc loss: 1.530, Gen loss: 1.346, Con loss: 3.748
[8/100] - time: 139.10, Disc loss: 1.444, Gen loss: 1.348, Con loss: 3.771
[9/100] - time: 139.02, Disc loss: 1.345, Gen loss: 1.375, Con loss: 3.818
[10/100] - time: 139.22, Disc loss: 1.243, Gen loss: 1.428, Con loss: 3.807
[11/100] - time: 139.17, Disc loss: 1.151, Gen loss: 1.483, Con loss: 3.853
[12/100] - time: 139.16, Disc loss: 1.087, Gen loss: 1.534, Con loss: 3.871
[13/100] - time: 139.03, Disc loss: 1.038, Gen loss: 1.582, Con loss: 3.910
[14/100] - time: 139.10, Disc loss: 0.998, Gen loss: 1.617, Con loss: 3.964
[15/100] - time: 139.16, Disc loss: 0.975, Gen loss: 1.636, Con loss: 3.954
[16/100] - time: 139.22, Disc loss: 0.958, Gen loss: 1.655, Con loss: 3.973
[17/100] - time: 139.14, Disc loss: 0.949, Gen loss: 1.660, Con loss: 4.040
[18/100] - time: 139.20, Disc loss: 0.942, Gen loss: 1.656, Con loss: 4.065
[19/100] - time: 139.05, Disc loss: 0.941, Gen loss: 1.643, Con loss: 4.090
[20/100] - time: 139.17, Disc loss: 0.930, Gen loss: 1.668, Con loss: 4.108
[21/100] - time: 139.10, Disc loss: 0.908, Gen loss: 1.684, Con loss: 4.146
[22/100] - time: 139.18, Disc loss: 0.899, Gen loss: 1.696, Con loss: 4.152
[23/100] - time: 139.03, Disc loss: 0.884, Gen loss: 1.705, Con loss: 4.110
[24/100] - time: 139.10, Disc loss: 0.870, Gen loss: 1.720, Con loss: 4.128
[25/100] - time: 139.18, Disc loss: 0.849, Gen loss: 1.745, Con loss: 4.138
[26/100] - time: 139.26, Disc loss: 0.841, Gen loss: 1.761, Con loss: 4.123
[27/100] - time: 139.12, Disc loss: 0.826, Gen loss: 1.763, Con loss: 4.178
[28/100] - time: 139.32, Disc loss: 0.817, Gen loss: 1.777, Con loss: 4.141
[29/100] - time: 139.14, Disc loss: 0.806, Gen loss: 1.813, Con loss: 4.146
[30/100] - time: 139.27, Disc loss: 0.783, Gen loss: 1.836, Con loss: 4.158
[31/100] - time: 139.18, Disc loss: 0.787, Gen loss: 1.833, Con loss: 4.151
[32/100] - time: 139.23, Disc loss: 0.767, Gen loss: 1.856, Con loss: 4.169
[33/100] - time: 139.12, Disc loss: 0.769, Gen loss: 1.845, Con loss: 4.158

Training with high resolution cartoon images

Thank you for your wonderful repository.
My question is, I want to use cartoon images that have very high resolution and detail (2K).
Is it a good idea to split/slice the images into smaller pieces (2x2 or 4x4) before training to get better results?
Any other suggestions are welcome.

Question about the weight of two losses

Hi , first of all , thanks for your share!
I want to ask about that the weight of two losses, adv loss and content loss, how do you set the balance of them? I found in your code, both of them you mul 10, but in the paper ,only the content loss mul 10.Could you tell me why make this change or where am I wrong?
Thanks again!

About patch-level discriminator

I noticed that you said "we use a simple patch-level discriminator ... in D",but I didn't find the counterpart in the code. Did I miss it? Or you just use the normal D without patch strategy,if this, how does it impact the performance? slightly or dramatically or nothing to hurt? :-)

关于Gram矩阵

Hi,论文里提了用8个 residual blocks 来做图像风格化,但是这真的很耗时,为什么当时没有考虑使用 gram 矩阵来做呢?谢谢~

关于模型的shape变换

hi,作者您好,我已经训练好了一版不错的模型,但是因为训练数据的尺寸都是256*256的,当我将模型的输入换为其他的尺寸时,效果会明显变差,您知道是什么原因吗?

关于生成图像色调的问题

您好,最近我在用pytorch复现cartoon的工作,但是我这边发现了一个生成图像色调偏黑的问题。这个是取决于风格图吗?请问一下,卡通图的选取有什么需要注意的吗?我是按照每隔一段时间抽取一帧的方法。

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.