Hi, thank you very much for this wonderful example for implementing GAN using Keras.
I have one question, based on what I understood from the theory that there are three stages GAN should be train with:

1. training D for ground truth data as input with label 1 as output
2. training D using G output as an input and label 0 as output
3. same as (2) but with freezing D weights and label 1 as D output
   my question is: I didn't see any freezing step in your code. So, is that deferent way of implementation? or I miss something?
   Thanks again

How to use the Trained GAN model to generate digit images?

I'm trying on the basis of your example make a generator of color images.
But missing something with the color dimension.
And get error:
number of input channels does not match corresponding dimension of filter, 1 != 3

Code
https://pastebin.com/TAm21NJg

I want to use this code in a loop, but I am getting a ValueError due to Tensorflow variable sharing problem. I am unable to fix it, can you help?

in
https://github.com/roatienza/Deep-Learning-Experiments/blob/master/Experiments/Tensorflow/GAN/dcgan_mnist.py
you compute the generator loss as:
a_loss = self.adversarial.train_on_batch(noise, y)
but this also trains the discriminator using only the fake samples.
shouldn't you freeze the discriminator weights for this part?

Loaded training data...
Traceback (most recent call last):
File "rnn_words.py", line 56, in
dictionary, reverse_dictionary = build_dataset(training_data)
File "rnn_words.py", line 49, in build_dataset
count = collections.Counter(words).most_common()
File "/usr/lib64/python2.7/collections.py", line 444, in init
self.update(iterable, **kwds)
File "/usr/lib64/python2.7/collections.py", line 526, in update
self[elem] = self_get(elem, 0) + 1
TypeError: unhashable type: 'list'

I searched the web but do not solve it, anyone can help me?

I was trying to implement https://github.com/roatienza/Deep-Learning-Experiments/blob/master/Experiments/Tensorflow/GAN/dcgan_mnist.py and it works fine. But I tried the exact same code without the wrapper class ie I wrote just one class DCGAN and tried to build D, G and AM in the same class through various functions. I came across very weird Keras errors like TypeError: __call__() missing 1 required positional argument: 'inputs' for the line

Also, I noticed that there's a lot of duct-tape code like

 if self.D:
    return self.D

This pattern is repeated whenever you create a D/G/AM model. Why is this. Is this related to the other questions I raised?

in
https://github.com/roatienza/Deep-Learning-Experiments/blob/master/Experiments/Tensorflow/GAN/dcgan_mnist.py
you compute the discriminator loss as:

images_fake = self.generator.predict(noise)
x = np.concatenate((images_train, images_fake))
y = np.ones([2*batch_size, 1])
y[batch_size:, :] = 0
d_loss = self.discriminator.train_on_batch(x, y)

y[batch_size:, :] = 0 is this wrong?I think it is y[batch_size:, :] = 1

How long does it take to run 10K iterations on a 2.7 GHZ intel Core i5 processor Mac book?

I was at 150 iterations after so many hours? Is there a problem with code, system or I need a GPU?

@roatienza how to use this code for own dataset or other dataset like celeba as feature extractor use the discriminator network?

When i use cifar 10 as input to the same code, i get this error
ValueError: number of input channels does not match corresponding dimension of filter, 1 != 3

Hello. Prof Atienza. You said here (Lecture 3, Slide 23) that we can only use gradient descent in optimizing deep learning models (particularly loss function). I understand that both closed-form solution and Newton's method are computationally painful to use, that non-iterative 1st order optimization is inefficient, and that convex optimization is also not an option since most deep learning models have multiple extremas. We are therefore left only with gradient descent.

Are there other approaches to optimization in deep learning?

Hi,
When I increase the size of the file I see I am getting memory Error.

Looks

content = np.array(content)

is not able to get more data into the buffer (100MB Text file).
is there a way (or) any suggestions to make this same program work for large amount of data

I just run this python program of DCGAN with keras, it seems work and after 20 mins, all outputs and numbers are fixed and NEVER changed again, it seems dead, but WHY? output is copied below:

48: [D loss: 7.971192, acc: 0.500000] [A loss: 0.000000, acc: 1.000000]
49: [D loss: 7.971192, acc: 0.500000] [A loss: 0.000000, acc: 1.000000]
50: [D loss: 7.971192, acc: 0.500000] [A loss: 0.000000, acc: 1.000000]
51: [D loss: 7.971192, acc: 0.500000] [A loss: 0.000000, acc: 1.000000]
52: [D loss: 7.971192, acc: 0.500000] [A loss: 0.000000, acc: 1.000000]
53: [D loss: 7.971192, acc: 0.500000] [A loss: 0.000000, acc: 1.000000]
54: [D loss: 7.971192, acc: 0.500000] [A loss: 0.000000, acc: 1.000000]
55: [D loss: 7.971192, acc: 0.500000] [A loss: 0.000000, acc: 1.000000]
56: [D loss: 7.971192, acc: 0.500000] [A loss: 0.000000, acc: 1.000000]
57: [D loss: 7.971192, acc: 0.500000] [A loss: 0.000000, acc: 1.000000]
58: [D loss: 7.971192, acc: 0.500000] [A loss: 0.000000, acc: 1.000000]
59: [D loss: 7.971192, acc: 0.500000] [A loss: 0.000000, acc: 1.000000]
60: [D loss: 7.971192, acc: 0.500000] [A loss: 0.000000, acc: 1.000000]
61: [D loss: 7.971192, acc: 0.500000] [A loss: 0.000000, acc: 1.000000]
62: [D loss: 7.971192, acc: 0.500000] [A loss: 0.000000, acc: 1.000000]
63: [D loss: 7.971192, acc: 0.500000] [A loss: 0.000000, acc: 1.000000]
64: [D loss: 7.971192, acc: 0.500000] [A loss: 0.000000, acc: 1.000000]
65: [D loss: 7.971192, acc: 0.500000] [A loss: 0.000000, acc: 1.000000]
66: [D loss: 7.971192, acc: 0.500000] [A loss: 0.000000, acc: 1.000000]
67: [D loss: 7.971192, acc: 0.500000] [A loss: 0.000000, acc: 1.000000]

thanks for your article (https://medium.com/towards-data-science/gan-by-example-using-keras-on-tensorflow-backend-1a6d515a60d0), i like it. But i have a question want to ask you:

Do we need a fixed discriminator when training the generator in dcgan_mnist.py?

because i think it need fixed.

thanks

AttributeError Traceback (most recent call last)
in ()
190
191 if name == 'main':
--> 192 mnist_dcgan = MNIST_DCGAN()
193 timer = ElapsedTimer()
194 mnist_dcgan.train(train_steps=10000, batch_size=256, save_interval=500)

in init(self)
135
136 self.DCGAN = DCGAN()
--> 137 self.discriminator = self.DCGAN.discriminator_model()
138 self.adversarial = self.DCGAN.adversarial_model()
139 self.generator = self.DCGAN.generator()

in discriminator_model(self)
111 optimizer = RMSprop(lr=0.0002, decay=6e-8)
112 self.DM = Sequential()
--> 113 self.DM.add(self.discriminator())
114 self.DM.compile(loss='binary_crossentropy', optimizer=optimizer, metrics=['accuracy'])
115 return self.DM

in discriminator(self)
47 input_shape = (self.img_rows, self.img_cols, self.channel)
48 self.D.add(Conv2D(depth*1, 5, strides=2, input_shape=input_shape, padding='same'))
---> 49 self.D.add(LeakyReLU(alpha=0.2))
50 self.D.add(Dropout(dropout))
51

E:\Anaconda3\envs\python35\lib\site-packages\keras\models.py in add(self, layer)
490 output_shapes=[self.outputs[0]._keras_shape])
491 else:
--> 492 output_tensor = layer(self.outputs[0])
493 if isinstance(output_tensor, list):
494 raise TypeError('All layers in a Sequential model '

E:\Anaconda3\envs\python35\lib\site-packages\keras\engine\topology.py in call(self, inputs, **kwargs)
617
618 # Actually call the layer, collecting output(s), mask(s), and shape(s).
--> 619 output = self.call(inputs, **kwargs)
620 output_mask = self.compute_mask(inputs, previous_mask)
621

E:\Anaconda3\envs\python35\lib\site-packages\keras\layers\advanced_activations.py in call(self, inputs)
44
45 def call(self, inputs):
---> 46 return K.relu(inputs, alpha=self.alpha)
47
48 def get_config(self):

E:\Anaconda3\envs\python35\lib\site-packages\keras\backend\tensorflow_backend.py in relu(x, alpha, max_value)
2919 """
2920 if alpha != 0.:
-> 2921 x = tf.nn.leaky_relu(x, alpha)
2922 else:
2923 x = tf.nn.relu(x)

Hello
I want to generate fake time series using Gan's networks, but I can't use your network. I wanted you to help me if possible. Thankful

Hi,
I have a question as the title when I was studying DCGAN.

I found some other DCGAN examples of keras and it seems they will set the discriminator's parameter non-trainable during training adversarial.
Then resume it to trainable once training only discriminator model again.

But it seems in your DCGAN code doesn't do the work.
(that means setting trainable=False and recompile the model)
Is that a bug or did I misunderstand the training process of DCGAN?

Thanks.