original definition is :
rnn_inputs = tf.placeholder(tf.float32, [batch_size, 22, 1000])
it should be :
rnn_inputs = tf.placeholder(tf.int32, [batch_size, 22])

Im getting :Trying to share variable rnn/multi_rnn_cell/cell_0/basic_lstm_cell/weights, but specified shape (1024, 2048) and found shape (595, 2048).

To verify whats wrong in mi code, i run the solutions notebook with no modification and it gets the same error.

In decoding_layer_infer() function, following is the description for a parameter

:param maximum_length: Maximum length of

Which I think is incomplete.

Error message:

ValueError: Attempt to reuse RNNCell <tensorflow.contrib.rnn.python.ops.core_rnn_cell_impl.BasicLSTMCell object at 0x107d602b0> with a different variable scope than its first use. First use of cell was with scope 'rnn/multi_rnn_cell/cell_0/basic_lstm_cell', this attempt is with scope 'rnn/multi_rnn_cell/cell_1/basic_lstm_cell'. Please create a new instance of the cell if you would like it to use a different set of weights. If before you were using: MultiRNNCell([BasicLSTMCell(...)] * num_layers), change to: MultiRNNCell([BasicLSTMCell(...) for _ in range(num_layers)]). If before you were using the same cell instance as both the forward and reverse cell of a bidirectional RNN, simply create two instances (one for forward, one for reverse). In May 2017, we will start transitioning this cell's behavior to use existing stored weights, if any, when it is called with scope=None (which can lead to silent model degradation, so this error will remain until then.)

In dlnd_tv_script_generation.ipynb file, I noticed the following code snppet.

`# Probabilities for generating words
probs = tf.nn.softmax(logits, name='probs')

# Loss function
cost = seq2seq.sequence_loss(
    logits,
    targets,
    tf.ones([input_data_shape[0], input_data_shape[1]]))`

So I am confused why still use logits when probs is available?

While attempting to train the model in the "ANNA KaRNNa.ipynb" notebook, the following error occurs:

Epoch 1/1  Iteration 1/178 Training loss: 4.4205 4.7214 sec/batch

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-29-049284c61b0f> in <module>()
     33                     model.initial_state: new_state}
     34             batch_loss, new_state, _ = sess.run([model.cost, model.final_state, model.optimizer], 
---> 35                                                  feed_dict=feed)
     36             loss += batch_loss
     37             end = time.time()

/home/carnd/anaconda3/envs/dl/lib/python3.5/site-packages/tensorflow/python/client/session.py in run(self, fetches, feed_dict, options, run_metadata)
    765     try:
    766       result = self._run(None, fetches, feed_dict, options_ptr,
--> 767                          run_metadata_ptr)
    768       if run_metadata:
    769         proto_data = tf_session.TF_GetBuffer(run_metadata_ptr)

/home/carnd/anaconda3/envs/dl/lib/python3.5/site-packages/tensorflow/python/client/session.py in _run(self, handle, fetches, feed_dict, options, run_metadata)
    912     # Validate and process feed_dict.
    913     if feed_dict:
--> 914       feed_dict = nest.flatten_dict_items(feed_dict)
    915       for feed, feed_val in feed_dict.items():
    916         for subfeed, subfeed_val in _feed_fn(feed, feed_val):

/home/carnd/anaconda3/envs/dl/lib/python3.5/site-packages/tensorflow/python/util/nest.py in flatten_dict_items(dictionary)
    185             "Could not flatten dictionary. Key had %d elements, but value had "
    186             "%d elements. Key: %s, value: %s."
--> 187             % (len(flat_i), len(flat_v), flat_i, flat_v))
    188       for new_i, new_v in zip(flat_i, flat_v):
    189         if new_i in flat_dictionary:

ValueError: Could not flatten dictionary. Key had 4 elements, but value had 1 elements. Key: [<tf.Tensor 'zeros:0' shape=(100, 512) dtype=float32>, <tf.Tensor 'zeros_1:0' shape=(100, 512) dtype=float32>, <tf.Tensor 'zeros_2:0' shape=(100, 512) dtype=float32>, <tf.Tensor 'zeros_3:0' shape=(100, 512) dtype=float32>], value: [array([[[[-0.01657983,  0.01909606, -0.03410228, ...,  0.01559156,
          -0.02688509,  0.0422806 ],
         [-0.05654193,  0.06184309,  0.02283844, ...,  0.01506532,
          -0.02460467,  0.05574171],
         [-0.00022013, -0.00454104, -0.01119577, ..., -0.0139764 ,
          -0.03528586, -0.02026432],
         ------------------------values removed for brevity------------------------------------
         ..., 
         [-0.02444562,  0.00272722,  0.00821447, ...,  0.0003518 ,
          -0.00852541, -0.00531745],
         [-0.00063715, -0.00432429,  0.00045606, ..., -0.00780605,
          -0.01090723,  0.00569954],
         [ 0.00280242, -0.0048506 ,  0.00048869, ...,  0.00024749,
          -0.00114548,  0.00407875]]]], dtype=float32)].

I have attempted to run this on my local machine as well as the AWS image provided in the Cloud Computing lesson. The same error occurs under both environments.

many of us has been struggling to make the template code compatible with tf 1.1. In a lot of cases this is actually non-trivial.

This issue has been raised multiple times in the forum and most of cases, we are asked to downgrade to tf 1.0. But that is not a reasonable solution to just ask people to stick to the older as everything else are evolving.

See title.

in the section "Sequence to Sequence Model":
We can now stat defining the functions that will build the seq2seq model.
the word "stat" should be "start"

There's information leakage problem in project 1 where entire data is first normalized and then splitted into training and test sets.

In dlnd_tv_script_generation.ipynb, I have to change
capped_gradients = [(tf.clip_by_value(grad, -1., 1.), var) for grad, var in gradients]
to
capped_gradients = [(tf.clip_by_value(grad, -1., 1.), var) for grad, var in gradients if grad is not None]
to get the code running, since one of the grad value is None.

In the solution Jupyter notebook for gan_mist, cell 14, line 7,
d_loss_fake = tf.reduce_mean( tf.nn.sigmoid_cross_entropy_with_logits(logits=d_logits_fake, labels=tf.zeros_like(d_logits_real)))
should have a zero labels tensor like the fake logits, not the real ones:
d_loss_fake = tf.reduce_mean( tf.nn.sigmoid_cross_entropy_with_logits(logits=d_logits_fake, labels=tf.zeros_like(d_logits_fake)))

It was just luck that the fake and real logit tensors have the same length, so the code manages to work as is.

Is get_accuracy() code was wrong:

if max_seq - target.shape[1]:
        target = np.pad(
            target,
            [(0,0),(0,max_seq - target.shape[1]),(0,0)],

as the target has Rank 2 not 3

should be

if max_seq - target.shape[1]:
        target = np.pad(
            target,
            [(0,0),(0,max_seq - target.shape[1])],

I tried to define multiple layers of LSTMs in Project 4 according to these lines from Sentiment_RNN lesson.

Stack up multiple LSTM layers, for deep learning

cell = tf.contrib.rnn.MultiRNNCell([drop] * lstm_layers)

this worked well in the lesson with Tensorflow 1.0

But in Tensorflow 1.1 warnes that this is not good! We should use this solution (linked in Project 4 notebook)
https://github.com/tensorflow/tensorflow/blob/6947f65a374ebf29e74bb71e36fd82760056d82c/tensorflow/docs_src/tutorials/recurrent.md#stacking-multiple-lstms

def lstm_cell():
return tf.contrib.rnn.BasicLSTMCell(lstm_size)
stacked_lstm = tf.contrib.rnn.MultiRNNCell(
[lstm_cell() for _ in range(number_of_layers)])

I think it would be great to mention this in the lessons notebook already.

is it intended accessing target_batch and batch_train_logits directly from get_accuracy()?

def get_accuracy(target, logits):
    """
    Calculate accuracy
    """
    max_seq = max(target.shape[1], logits.shape[1])
    if max_seq - target.shape[1]:
        target = np.pad(
            target_batch,
            [(0,0),(0,max_seq - target_batch.shape[1]), (0,0)],
            'constant')
    if max_seq - batch_train_logits.shape[1]:
        logits = np.pad(
            logits,
            [(0,0),(0,max_seq - logits.shape[1]), (0,0)],
            'constant')

    return np.mean(np.equal(target, np.argmax(logits, 2)))

Since we are using batch normalization, tf.control_dependencies should be used here:
with tf.control_dependencies(tf.get_collection(tf.GraphKeys.UPDATE_OPS)):
d_train_opt = tf.train.AdamOptimizer(learning_rate, beta1=beta1).minimize(d_loss, var_list=d_vars)
g_train_opt = tf.train.AdamOptimizer(learning_rate, beta1=beta1).minimize(g_loss, var_list=g_vars)

Source:
https://github.com/udacity/deep-learning/blob/master/batch-norm/Batch_Normalization_Lesson.ipynb

Hi 👋

I found a small typo in the problem_unittests.py file (https://github.com/udacity/deep-learning/blob/master/language-translation/problem_unittests.py#L89).

assert max_target_sequence_length.op.type == 'Max', \
            'Max Target Sequence Length is not a Placeholder.'

Instead of == 'Max', we might use == 'Placeholder',.

What do you think?

Thank you in advance

Hi! Floydhub just updated and now the cpde provided to run the notebook on floydhub does not work

function build_rnn(cell, inputs) also need initial_state as a parameter for the tf.nn.dynamic_rnn. Otherwise we cannot feed the state to the network.

def build_rnn(cell, inputs, initial_state):
outputs, state = tf.nn.dynamic_rnn(cell, inputs, initial_state = initial_state)

build_nn() miss a parameter, batch_size

def build_nn(cell, rnn_size, input_data, vocab_size, embed_dim, batch_size):
cell, initial_state = get_init_cell(batch_size, rnn_size)

Up until now we've seen that, when using tf.layers.batch_normalization, we set it's second parameter to True or False depending on whether we are training or not. However, I noticed that in the DCGAN class this parameter is always True since we are not passing any specific value when running the session (so it keeps the default value specified in the method definition). Is there any reason for this or is this an issue?

Missing files: reviews.txt and labels.txt

Hi,

The updated repo for the release of new content (mainly Reinforcement Learning lessons) has an empty 'gym' directory. For the lesson, we need to install OpenAI which is supposed to be contained in that directory. Therefore, pip install -e . fails since there is no script in this directory.

after this change b5d76ad#diff-d449397f895434a819eb879dd0ca8f5cR639

It is not possible to execute the cell anymore as the variable embed_dim is not defined.

For the embedding you're using a shape = (n_words, embed_size). But in the preprocessing step we shift the indices of all the words by 1 and leave index 0 as the "null" word for padding. so wouldnt we need the shape to be (n_words + 1, embed_size).

in a toy example if we had a 10 word vocabulary we would need an 11 row embed matrix (with the 0th row serving as the "null" word). And all of the other words would be indexed 1 - 10.

In the data preprocessing, the solution used open(), read(), join(), split(), to get the reviews and labels. The problem is that it creates an empty review and label as the last row. Although reviews_ints = [each for each in reviews_ints if len(each) > 0] Filter out that review with 0 length, but features = np.zeros((len(reviews), seq_len), dtype=int) and subsequent iteration put the empty one back as the last forgotten row. Luckily, it is the last one. So it only causes an error of 0.04% for the testing accuracy.

The origin of this mistake is due to split('\n'). So keep in mind to manually remove the last element.

Alternatively, csv module does the job worry-free.

from string import punctuation
import csv
reviews = []
words =[]
with open('reviews.txt') as f:
    rows = csv.reader(f)
    for row in rows:
        text = ''.join([char for char in str(row) if char not in punctuation])
        words.extend(text.split())
        reviews.append(text)

In project 4 , in Decoding - Training section , it's written

Create a training decoding layer:
Create a tf.contrib.seq2seq.TrainingHelper
Create a tf.contrib.seq2seq.BasicDecoder
Obtain the decoder outputs from tf.contrib.seq2seq.dynamic_decode

but it is giving error module 'tensorflow.contrib.seq2seq' has no attribute 'TrainingHelper'

what should I do ?

just clone the TensorBoard project and I want to runabout something wrong.

//Write out the graph for TensorBoard
model = build_rnn(len(vocab),
                  batch_size=batch_size,
                  num_steps=num_steps,
                  learning_rate=learning_rate,
                  lstm_size=lstm_size,
                  num_layers=num_layers,
                  sampling=sampling,
                  grad_clip=grad_clip
                 )

with tf.Session() as sess:
    
    sess.run(tf.global_variables_initializer())
    file_writer = tf.summary.FileWriter('./logs/1', sess.graph)

the build_rnn run fail with these wrong:

ValueError                                Traceback (most recent call last)
<ipython-input-63-f61b00ac8e27> in <module>()
      6                   num_layers=num_layers,
      7                   sampling=sampling,
----> 8                   grad_clip=grad_clip,
      9                  )
     10 

I don't know what is wrong?

On line 34 of problem_unittests.py
_assert_tensor_shape(tensor, shape, 'Real Input')
should be
_assert_tensor_shape(tensor, shape, display_name)

In d_loss_fake = tf.reduce_mean( tf.nn.sigmoid_cross_entropy_with_logits(logits=d_logits_fake, labels=tf.zeros_like(d_logits_real)))

Since we use fake logits for labels of all zeros for discriminator, should we pass d_logits_fake in for the labels, e.g. ... labels=tf.zeros_like(d_logits_fake)...?

in first description of GANs
As the generator learns through training, it figures out how to map these random vectors to recognizable images that can foold the discriminator.

In the embeddings get_batches function there is this line

# only full batches
words = words[:n_batches*batch_size]

n_batches is the length of words so it seems this extends words to be 1000x it's size.

What is this trying to do?

I believe the current wording in the embeddings (word2vec) exercise can be changed to avoid confusions.

embeddings/Skip-Gram%20word2vec.ipynb says ("Building the graph" cell):

We can discard the softmax layer becuase we don't really care about making predictions with this network.

Later on, in the "Negative Sampling" cell:

For every example we give the network, we train it using the output from the softmax layer.

Hi, I'm working on Sentiment_Classification_Projects.ipynb.

And I am confused with following code at the end of project 3:
mlp = SentimentNetwork(reviews[:-1000],labels[:-1000], learning_rate=0.1)

It will preprocess the given reviews to fill word2index which is a dict, but when I execute next line:
mlp.test(reviews[-1000:],labels[-1000:])
It throws KeyError: 'reprogram'.

It seems that there are some words only existing in last 1000 reviews, and I think first line should be:
mlp = SentimentNetwork(reviews, labels, learning_rate=0.1)

I'm not sure about it, did I miss something?

in the "populate the memory section"

Make a random action

action = env.action_space.sample()
next_state, reward, done, _ = env.step(action)

it would seem that next_state should be state

original definition is:
dec_embed_input = tf.placeholder(tf.float32, [batch_size, 22, embedding_size])
it should be:
dec_embed_input = tf.placeholder(tf.int32, [batch_size, sequence_length])

In the 'Build the Neural Network > Build the Neural Network' section, the instructions are to use process_encoding_input to process the target data. At the beginning of the Build the Neural Network section, this function is also listed as one of the functions we have to implement. The rubric also mentions this function.

I'm a bit confused - do the instructions need to be updated or is there a section (with instructions to build this function) for this missing?

fwiw, this section was present in an earlier version of the notebook

Hi i tried the solution for transfer learning

codes = codes.reshape((len(labels), -1))
got the error

i got this i reduced the number of images as a poc on my PC.

running windows and 3.5.2 python cpu no gpus.

i added print (len(codes )) to verify info.

in ()
10 codes = np.fromfile(f, dtype=np.float32)
11 print(len(codes))
---> 12 codes = codes.reshape((len(labels),-1))

ValueError: cannot reshape array of size 9187328 into shape (2244,newaxis)

When attempting to continue from the Checkpoint (immediately prior to checking for gpu) then unit tests are not imported and the first code cell in the notebook must still be run.

It was said to be here. Where is it?

I ran into this error while running the script. I didn't modify anything.

How should this problem be solved ?

How do you apply model built in “Sentiment Prediction RNN” lesson to get sentiment out of some example review you write yourself? So you can later use it in production.

Hi,
Issue report of Anna KaRNNa.ipynb in intro-to-rnns.
I got an error when this code tries to save checkpoints during training.

Code
saver.save(sess, "checkpoints/i{}_l{}_v{:.3f}.ckpt".format(iteration, lstm_size, np.mean(val_loss)))

Error message

• saver.py in save(self, sess, save_path, global_step, latest_filename, meta_graph_suffix, write_meta_graph, write_state)
• 1363 if not gfile.IsDirectory(os.path.dirname(save_path)):
• 1364 raise ValueError(
• -> 1365 "Parent directory of {} doesn't exist, can't save.".format(save_path))
• 1366
• 1367 save_path = os.path.dirname(save_path)
• ValueError: Parent directory of checkpoints/i2_l512_v3.780.ckpt doesn't exist, can't save.

To solve this issue, I created "checkpoints" directory under intro-to-rnns with mkdir command manually. Thank you.

Satoshi

Since the python version we use is already equal or higher than 3.5, including type hint in projects code should enhance the IDE's auto-complete function and avoid lots of type errors while we writing code, as well as more clear indicating the requirement of every function we should write.

Type hint feature refers: https://www.python.org/dev/peps/pep-0484/

When I run conda env create -f ./environments/dl_env_mac.yml on Mac, I get error output:

Using Anaconda API: https://api.anaconda.org
Fetching package metadata .........

NoPackagesFoundError: Package missing in current osx-64 channels:

• tornado 4.5 py36_0

When I run conda search tornado, it seems that there's no 4.5 version of tornado at all.

EOFError: Ran out of input.
Not able to solve this error .

The "Testing" Block (Last block) does not seem to restore correctly:
saver.restore(sess, tf.train.latest_checkpoint('/output/checkpoints'))

Probably because the Training Block (previous block) saves training data like this:
saver.save(sess, "checkpoints/sentiment.ckpt")

I changed the Testing block to the following and it appears to work:
saver.restore(sess, tf.train.latest_checkpoint('checkpoints'))

I'm not sure but this line in the unit test doesn't look right. The shape of inf_output isn't being tested.

assert train_output.get_shape().as_list() == [batch_size, None, vocab_size],\
            'Train Logits is the wrong shape: {}'.format(train_output.get_shape())
assert train_output.get_shape().as_list() == [batch_size, None, vocab_size], \
            'Inference Logits is the wrong shape: {}'.format(train_output.get_shape())

In 'build the graph',
max_target_sentence_length = max([len(sentence) for sentence in source_int_text])
is meant to be
.. for sentence in target_int_text])?

in the process decoding input section ".... each batch in target_data and concat the GO ID to the ..." perhaps GO should be replaced with

Would be nice to have, so when running the notebooks I don't have to install as a go through cells individually.