gauravmm / cifar-gan Goto Github PK

Label Smoothing, i.e. if you have two target labels: Real=1 and Fake=0, then for each incoming sample, if it is real, then replace the label with a random number between 0.7 and 1.2, and if it is a fake sample, replace it with 0.0 and 0.3 (for example).
Salimans et. al. 2016
make the labels the noisy for the discriminator: occasionally flip the labels when training the discriminator

weights/, train_logs/ not automatically created.

Hey,

While running autorun.sh I get the following error :
if [ ! -d .git] ; then (line 6)
raises an invalid syntax error

Probably another Python 3.4 vs Python 3.5 error ?

Weights take up a lot of disk space.

Instead of the current fixed ratio, allow the generator and discriminator to be trained based on loss function and batch count.

I get the following error message :"Parent module '.' not loaded cannot perform relative import" when calling adversarial.py script.

Apparently this line of support (l.163) is responsible for the error:
return lambda modname: importlib.import_module(pkg + "." + modname, package=".")

Perhaps I don't have a module allowing the import of modules when stacking names ?

Cheers,

This is good to demonstrate the progress of training.

Forked from #23

https://github.com/soumith/ganhacks

TODO: Update calls to Keras 2 API to remove warnings. We may have to restructure the code accordingly (TODO in adversarial.py)

Hey,

I receive the following error when trying to compile the experiment :
"module" object has no attribute "label_flipping_prob"
related to support.py line 52:
self.label_dis_real = _selection_stream([args.hyperparam.label_flipping_prob]
And I don't understand the purpose of this attribute of args.hyperparam

We need to:

1. Allow a commandline option to override the data prefix.
2. Move all the same run data into a single subfolder.
3. Use the preprocessors and the hyperparameters as part of the naming convention.

Off by one error, so batch number is one less than reported.

Construct different mini-batches for real and fake, i.e. each mini-batch needs to contain only all real images or all generated images.
when batchnorm is not an option use instance normalization (for each sample, subtract mean and divide by standard deviation).

Hey guys,

I face the following issue, due to my version of Python being 3.4 whereas the initial code was written with Python 3.5.
In the def random_stream function line 91 of support, I had to modify *img_size, as starred expressions are not accepted by Python 3.4. I wrote img_size[0], img_size[1], img_size[2] instead.

Use mypy to enforce type correctness.

if you have labels available, training the discriminator to also classify the samples: auxillary GANs

2017-03-30 12:28:57, 7, [ 0.83778523  0.        ], [ 0.75879407  0.        ], [ 0.6237818  0.       ]
2017-03-30 12:29:00, 8, [ 0.90475029  0.        ], [ 0.77940774  0.        ], [ 0.58533067  0.        ]
2017-03-30 12:29:03, 9, [ 0.98412164  0.        ], [ 0.78705633  0.        ], [ 0.55018032  0.        ]
2017-03-30 12:29:06, 10, [ 1.12536469  0.        ], [ 0.80081189  0.        ], [ 0.49073929  0.        ]
2017-03-30 12:29:09, 11, [ 1.35740131  0.        ], [ 0.81628722  0.        ], [ 0.43616527  0.        ]

We need to change the output to follow the CSV format.

Provide noise in the form of dropout (50%).
Apply on several layers of our generator at both training and test time
https://arxiv.org/pdf/1611.07004v1.pdf

Dont try to find a (number of G / number of D) schedule to uncollapse training
It's hard and we've all tried it.
If you do try it, have a principled approach to it, rather than intuition

Add some artificial noise to inputs to D (Arjovsky et. al., Huszar, 2016)
http://www.inference.vc/instance-noise-a-trick-for-stabilising-gan-training/
https://openreview.net/forum?id=Hk4_qw5xe
adding gaussian noise to every layer of generator (Zhao et. al. EBGAN)
Improved GANs: OpenAI code also has it (commented out)

D loss goes to 0: failure mode
check norms of gradients: if they are over 100 things are screwing up
when things are working, D loss has low variance and goes down over time vs having huge variance and spiking
if loss of generator steadily decreases, then it's fooling D with garbage (says martin)

Observed on Python 3.5.2, Ubuntu 16.10.

Use this documentation to implement a custom metric: https://keras.io/metrics/#custom-metrics

Currently, headers are written as:

time, batch, composite loss, discriminator+real loss, discriminator+fake loss

But data is written as:

2017-03-20 09:07:36s, 50d, [ 0.70245182  0.56332031]f, [ 0.70011313  0.46328125]f, [ 0.7290282   0.26015625]f

The [a, b] sections need to (1) be unwrapped and (2) have appropriate titles inserted.

In GAN papers, the loss function to optimize G is min (log 1-D), but in practice folks practically use max log D

because the first formulation has vanishing gradients early on
Goodfellow et. al (2014)
In practice, works well:

Flip labels when training generator: real = fake, fake = real

Currently, we report 12 metrics ([loss, acc, lbl_true, lbl_false] x [labelled, unlabelled, composite]). We can do away with some of these, and reorder the rest accordingly.

the stability of the GAN game suffers if you have sparse gradients
LeakyReLU = good (in both G and D)
For Downsampling, use: Average Pooling, Conv2d + stride
For Upsampling, use: PixelShuffle, ConvTranspose2d + stride
PixelShuffle: https://arxiv.org/abs/1609.05158

Use DCGAN when you can. It works!
if you cant use DCGANs and no model is stable, use a hybrid model : KL + GAN or VAE + GAN

Experience Replay
Keep a replay buffer of past generations and occassionally show them
Keep checkpoints from the past of G and D and occassionaly swap them out for a few iterations
All stability tricks that work for deep deterministic policy gradients
See Pfau & Vinyals (2016)

optim.Adam rules!
See Radford et. al. 2015
Use SGD for discriminator and ADAM for generator

Produced real and fake labels are swapped.

Hey,

I pulled the latest code and got an error with default.py script at line 63:
"MovingAverage" is not defined...

Dont sample from a Uniform distribution

Sample from a gaussian distribution

When doing interpolations, do the interpolation via a great circle, rather than a straight line from point A to point B
Tom White's Sampling Generative Networks ref code https://github.com/dribnet/plat has more details

The current losses are too volatile to be useful. Report a sliding-window average loss instead.

Wrap the data source and transformations properly. (TODO in adversarial.py)