Hi,

In the function conv() in DCGAN notebook, when defining the conv_layer, by default you create a conv_layer with no bias no matter with or without batch norm. I think you should put that conv_layer in if-else statement so that we have the bias in the conv_layer when we don't use batch norm.
if batch_norm: conv_layer = ...**bias=False** else **bias=True**

The markdown cell (number 5) titled "TODO: Implementing the basic functions" has inverted order of y and y_hat in the "The function that updates the weights" formula. It should be

instead of

A couple things I noticed about the "Analyzing Student Data" notebook:

1. In the first plot, there is a small bug and it's not showing up. It just shows the metadata on the plot object. Adding "%matplotlib inline" before import matplotlib seems to fix this (credits to MSJose in the Slack channel).
2. The sigmoid_prime function is not used anywhere. It should not be there since we calculate the derivative directly in the error_term_formula function.
3. There is a typo in the train_nn function. Since we declare the error equation as -(y - y_hat)*sigmoid_prime(x), when we update the gradient it should be del_w -= error_term * x. The solution gets around this by removing the negative sign in the error equation. My point here is that if we follow the error equation in the notebook which has the negative sign, the training won't work with the way the weights are updated currently in the train_nn function.
   Another way we could fix this without changing the test function is to get rid of the negative sign in the error equation.

Hello ,

The following code snippet below raises an error when executed;

# Grab some data 
dataiter = iter(trainloader)
images, labels = dataiter.next()

# Resize images into a 1D vector, new shape is (batch size, color channels, image pixels) 
images.resize_(64, 1, 784)
# or images.resize_(images.shape[0], 1, 784) to automatically get batch size

# Forward pass through the network
img_idx = 0
ps = model.forward(images[img_idx,:])

img = images[img_idx]
**helper.view_classify(img.view(1, 28, 28), ps)**

Error:

AttributeError Traceback (most recent call last)
in ()
12
13 img = images[img_idx]
---> 14 helper.view_classify(img.view(1, 28, 28), ps)

AttributeError: module 'helper' has no attribute 'view_classify'

Please suggest.

Regards
Surojit

Hi,

In the CycleGAN solution notebook, in the Discriminator architecture, as I have seen the image of the architecture, the dimension of the last output (the logit) is 1x1x1; in the following code from the Discriminator:

        self.conv5 = conv(conv_dim*8, 1, 4, stride=1, batch_norm=False)

and I see that the output from self.conv4 is 8x8x512, if it goes through self.conv5, the output should be of shape 7x7x1. How can it be 1x1x1 as you defined in the image architecture as well as in the video lecture (you said it should output a single value)?

Potential improvement in init_network function under class SentimentNetwork:
To initialize weights between the hidden layer and output layer, self.output_nodes**-0.5 should be changed to self.hidden_nodes**-0.5.

Part 3 - Training Neural Networks (Exercises).ipynb

Would you please put a random seed at the top of each notebook so that we can check our results against the Solutions .ipynb?

Eg:

torch.manual_seed(1)
if torch.cuda.is_available:
    torch.cuda.manual_seed_all(1)

The 'Part 1' notebook had this (but only for CPU). None of the following ones do.

In the section on nn.LogSoftmax, the exercise has the variable name logits, but the solution changes it to logps - it my be worthwhile updating the exercise to match.

In the looped network at the end, you have an else: after a for-loop with no break, which is unnecessary.

This notebook calls model.forward(images) twice.

This says that .forward() should be avoided as hooks don't run:
https://discuss.pytorch.org/t/any-different-between-model-input-and-model-forward-input/3690

In the final code block, softmax() is used when the instructions above say to use exp() if the final model operation is LogSoftmax()

I would make a PR for these myself, but am lacking the time. Perhaps another motivated student wishes to do so.

The coursework describes how to download the latest project files from this repo, but these projects do not contain the .udacity_pa file therefore unable to submit them using udacity submit.

The error term formula is given as (y-yhat)*sigmoid_prime(x). Shouldn't it be (y-yhat)/(yhat(1-yhat)) * sigmoid_prime(x)? Isn't it derived from this chain rule application dE/dyhat * dyhat/dh?

The fc_model.py file is not included in the folder. This prevents this notebook from executing without error. The module can be found here

Numpy to Torch and back

import numpy as np
a = np.random.rand(4,3)
a

this solves the issue

Hi,
In 'Part1 - Tensors in Pytorch (Exercises).ipynb', 'Part 1 - Tensors in PyTorch (Solution).ipynb', I think there is a typing mistake.

Remember that for matrix multiplications, the number of columns in the first tensor must equal to the number of rows in the second *column*.

--> Remember that for matrix multiplications, the number of columns in the first tensor must equal to the number of rows in the second *tensor*.

Hi, in the RNN sentiment analysis solution (https://github.com/udacity/deep-learning-v2-pytorch/blob/master/sentiment-rnn/Sentiment_RNN_Solution.ipynb), when I use the predict() function provided in the notebook with negative reviews (even the ones in the training/valid/test data), I don't get the expected result most of the time (accuracy: 1/4). Here is the code I added in a cell at the end of the notebook:

# from https://github.com/udacity/deep-learning-v2-pytorch/blob/f456d6167d8af205154cd27f0bcfe5eb813e2503/sentiment-rnn/Sentiment_RNN_Solution.ipynb
# at the end of the notebook...

rev1= 'this film is one giant pant load . paul schrader is utterly lost in his own bad screenplay . and his directing is about as comatose as it can be without his actually having been sleepwalking during the process .  br    br   the worst though is woody harrelson  whom i ordinarily like when he  s properly cast . he plays  the walker   a homosexual man in d . c . who plays social companion to the bored wives of the washington elite . he couldn  t have been more one dimensional if he had been cut out of a magazine and bounced around in front of the camera on a popsicle stick . his  southern accent  is that  off the rack  version that decrescendos from the beginning to the end of every line he delivers  as though the heat and humidity of the south is still draining him of every ounce of energy he has . it is monotonous . but  his is not the worst accent in the movie . his  boyfriend   played by moritz bleibtreau  attempts to affect some kind of a mid east accent that is so clumsy he can barely deliver the bad lines written for him . he is incapable of rolling his r  s in spite of the fact that in real life he is german  and speaks several languages  one of them being italian  that  s kind of a good reason to cast someone else don  t ya think   br    br   from the story  to the screenplay  to the directing  to the camera work  to the performances by the leads  this movie is bad from beginning to end . the only tolerable moments in this film came from three supporting actresses lily tomlin  lauren bacall  and kristin scott thomas . only these three managed to make it through this movie with their dignity in tact . in fact  all three are excellent  in spite of being trapped in a really bad film . ufortunately  no one could ever be good enough to redeem this endless series of flaws . if you like these three actresses  watch them in something else . this movie is not worth your time .'
rev2 = "when i was little my parents took me along to the theater to see interiors . it was one of many movies i watched with my parents  but this was the only one we walked out of . since then i had never seen interiors until just recently  and i could have lived out the rest of my life without it . what a pretentious  ponderous  and painfully boring piece of    s wine and cheese tripe . woody allen is one of my favorite directors but interiors is by far the worst piece of crap of his career . in the unmistakable style of ingmar berman  allen gives us a dark  angular  muted  insight in to the lives of a family wrought by the psychological damage caused by divorce  estrangement  career  love  non  love  halitosis  whatever . the film  intentionally  has no comic relief  no music  and is drenched in shadowy pathos . this film style can be best defined as expressionist in nature  using an improvisational method of dialogue to illicit a  more pronounced depth of meaning and truth  . but woody allen is no ingmar bergman . the film is painfully slow and dull . but beyond that  i simply had no connection with or sympathy for any of the characters . instead i felt only contempt for this parade of shuffling  whining  nicotine stained  martyrs in a perpetual quest for identity . amid a backdrop of cosmopolitan affluence and baked brie intelligentsia the story looms like a fart in the room . everyone speaks in affected platitudes and elevated language between cigarettes . everyone is  lost  and  struggling   desperate to find direction or understanding or whatever and it just goes on and on to the point where you just want to slap all of them . it  s never about resolution  it  s only about interminable introspective babble . it is nothing more than a psychological drama taken to an extreme beyond the audience  s ability to connect . woody allen chose to make characters so immersed in themselves we feel left out . and for that reason i found this movie painfully self indulgent and spiritually draining . i see what he was going for but his insistence on promoting his message through prozac prose and distorted film techniques jettisons it past the point of relevance . i highly recommend this one if you  re feeling a little too happy and need something to remind you of death . otherwise  let  s just pretend this film never happened ."
rev3 = "this film lacked something i couldn  t put my finger on at first charisma on the part of the leading actress . this inevitably translated to lack of chemistry when she shared the screen with her leading man . even the romantic scenes came across as being merely the actors at play . it could very well have been the director who miscalculated what he needed from the actors . i just don  t know .  br    br   but could it have been the screenplay  just exactly who was the chef in love with  he seemed more enamored of his culinary skills and restaurant  and ultimately of himself and his youthful exploits  than of anybody or anything else . he never convinced me he was in love with the princess .  br    br   i was disappointed in this movie . but  don  t forget it was nominated for an oscar  so judge for yourself ."
rev4 = "sorry everyone    i know this is supposed to be an  art  film   but wow  they should have handed out guns at the screening so people could blow their brains out and not watch . although the scene design and photographic direction was excellent  this story is too painful to watch . the absence of a sound track was brutal . the loooonnnnng shots were too long . how long can you watch two people just sitting there and talking  especially when the dialogue is two people complaining . i really had a hard time just getting through this film . the performances were excellent  but how much of that dark  sombre  uninspired  stuff can you take  the only thing i liked was maureen stapleton and her red dress and dancing scene . otherwise this was a ripoff of bergman . and i  m no fan f his either . i think anyone who says they enjoyed     hours of this is   well  lying . "

# seq_length = 200
print("rev1", predict(net, rev1, seq_length))
print("rev2", predict(net, rev2, seq_length))
print("rev3", predict(net, rev3, seq_length))
print("rev4", predict(net, rev4, seq_length))

# output:
# rev1 1
# rev2 0
# rev3 1
# rev4 1
# mostly ones...

(also available here: https://gist.github.com/saadtazi/7e641645fc90b3d547165ce4e2f10800)

Any idea why?

I am using colab and already !pip install helper , but I facing the error AttributeError: module 'helper' has no attribute 'imshow' and also
AttributeError: module 'helper' has no attribute 'view_classify'
What is wrong with helper??

Hi,

In the CycleGan_solution notebook, in function forward() in class ResidualBlock, why don't we apply the relu activation function to the out_2 - the output of each residual block? As I have seen in the image in the description, it says that we have the addition first F(x) + x, followed by a relu activation function.

In Part 6, the location given is:

'F_MNIST_data/'

However, in Part 4, it is:

'~/.pytorch/F_MNIST_data/'

This wastes disk space and is inconsistent.

The notebook "Part 8 - Transfer Learning (Exercises).ipynb" in "into-to-pytorch" folder has the TODO exercise about transformations already solved, even though it is not a "solution" notebook.

'Part 5 - Inference and Validation (Exercises).ipynb'

Replace:

accuracy = torch.mean(equals.type(torch.FloatTensor))

with:

accuracy = torch.mean(equals.float())

For the test set accuracy, why bother with the exponentiation:

ps = torch.exp(log_ps)
top_p, top_class = ps.topk(1, dim=1)

Why not just:

top_p, top_class = log_ps.topk(1, dim=1)

I have a question about model.train() and model.eval().

As far as I see in lecture content video by Cezanne together with the notebooks, for example, "mnist_mlp_solution_with_validation" or "cifar10_cnn_augmentation", why don't we use model.eval() for the validation and test time and model.train() for training? We have dropout in the model, so I think we should switch between 2 modes.

In the lecture video of "Deep learning with pytorch" by Mat Leonard, in notebook "Part 5 - Inference and Validation" in this link https://github.com/udacity/DL_PyTorch, I see that Mat switches model.train() and model.eval() during training and validating. I asked my mentor and he asked me to raise an issue here.

Thank you.

At the StudentAdmission notebook: it said that "Use the get_dummies function in numpy in order to one-hot encode the data" ,but I it's available on Pandas

Add information and example about operator "@" which override for matrix product in numpy and pytorch.
Operation: torch.mm(x, y) equal x @ y. For numpy same: np.matmul(a, b) equal a @ b.
This notation not common but you can see it sometime.
And some trick for weights.view(5,1) you can get using notation: weights.t() - shorter.

When not using CUDA, there is a type error. It might be due to PyTorch better reducing input data prior to training, and so the target specified for the CrossEntropy / NLL Loss needs to be cast to torch.LongTensor as opposed to the torch.IntTensor that was being input as the target for the loss function

the minimal necessary changes in the module train:

loss = criterion(output, targets.view(batch_size*seq_length).type(torch.LongTensor))

val_loss = criterion(output, targets.view(batch_size*seq_length).type(torch.LongTensor))

The images that are to be displayed under the "Activation Functions" and "Your Turn to Build a Network" are not displayed. Rather the code is displayed there.

Hi,

In the Sentiment_RNN_Solution, in the forward() when building the network, in the part:
# reshape to (hidden_dim, batch_size, n_classes)
sig_out = sig_out.view(-1, batch_size, self.output_size)
sig_out = sig_out[0] # get last batch of labels*

Instead of reshaping and pick the last batch like in the solution above, why don't we just use sig_out = sig_out[ : , -1, : ] with only 1 line. This picks the sigmoid output from the end of each batch.

In addition, as I have seen the output size after the self.fc(lstm_out), the size is (batch=50, seq_len=200, output=2), if we reshape using sig_out.view(-1, batch_size, self.output_size), how can we get hidden_dim be the first dimension like in the lecture as well as in the notebook's comment? And when we reshape like that, although we get the correct size (seq_len, batch_size, output_size), the order of numbers will be mixed up and wrong, this just reshapes to rows by rows. I mean, before reshaping, for each batch of 50, there are 200x2 values corresponding to that batch. Then we just use normal reshape, we get size of (seq_len, batch_size, output_size), but now the sequences and batches are not related to each other anymore, their values are mixed now. We need some method similar like transpose or numpy array.T in order to conserve their corresponding values.

By the way, suppose we can get the correct size (seq_len, batch, n_classes=2) with their corresponding values after resizing, why do we want to use sig_out[0]? In this case, sig_out[-1] is correct because we just want to select the sigmoid output from the last word of each batch.

Hi,

In the notebook "Skip_Grams" of word2vec, in the function cosine_similarity(), you miss the magnitudes of vector a in order to calculate cosine similarity.

In the code below, the magnitudes here are just magnitudes of b only. We need to add magnitude of vector a.
similarities = torch.mm(valid_vectors, embed_vectors.t())/magnitudes

Hi,
In Part 2 - Neural Networks in PyTorch (Exercises).ipynb, inside the second code cell, the following code works fine, but, logically wrong

transform = transforms.Compose([transforms.ToTensor(),
                              transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5)),
                              ])

it should be

transform = transforms.Compose([transforms.ToTensor(),
                              transforms.Normalize((0.5,), (0.5, )),
                              ])

reason being that MNIST has only one color channel and what you mentioned is for an image with 3 channels

in this solution notebook, you are freezing layers using this code:

# Freeze training for all "features" layers
for param in vgg16.features.parameters():
    param.require_grad = False 

but I think param.requires_grad should be used instead of param.require_grad (the s in param.requires_grad is missing).

Here is the pytorch documentation that explains that property for reference.

I believe this is a windows related problem, the following error occurs when running the solution solution notebook:

RuntimeError: Expected tensor for argument #1 'indices' to have scalar type Long; but got CUDAIntTensor instead (while checking arguments for embedding)

The full traceback:

---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-130-9f7dea11cb7b> in <module>
     32 
     33         # get the output from the model
---> 34         output, h = net(inputs, h)
     35 
     36         # calculate the loss and perform backprop

c:\program files\python36\lib\site-packages\torch\nn\modules\module.py in __call__(self, *input, **kwargs)
    489             result = self._slow_forward(*input, **kwargs)
    490         else:
--> 491             result = self.forward(*input, **kwargs)
    492         for hook in self._forward_hooks.values():
    493             hook_result = hook(self, input, result)

<ipython-input-123-f3aa96ff9410> in forward(self, x, hidden)
     30         batch_size = x.size(0)
     31 
---> 32         embeds = self.emb(x)
     33         lstm_out, hidden = self.lstm(embeds, hidden)
     34 

c:\program files\python36\lib\site-packages\torch\nn\modules\module.py in __call__(self, *input, **kwargs)
    489             result = self._slow_forward(*input, **kwargs)
    490         else:
--> 491             result = self.forward(*input, **kwargs)
    492         for hook in self._forward_hooks.values():
    493             hook_result = hook(self, input, result)

c:\program files\python36\lib\site-packages\torch\nn\modules\sparse.py in forward(self, input)
    106         return F.embedding(
    107             input, self.weight, self.padding_idx, self.max_norm,
--> 108             self.norm_type, self.scale_grad_by_freq, self.sparse)
    109 
    110     def extra_repr(self):

c:\program files\python36\lib\site-packages\torch\nn\functional.py in embedding(input, weight, padding_idx, max_norm, norm_type, scale_grad_by_freq, sparse)
   1074         with torch.no_grad():
   1075             torch.embedding_renorm_(weight, input, max_norm, norm_type)
-> 1076     return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)
   1077 
   1078 

RuntimeError: Expected tensor for argument #1 'indices' to have scalar type Long; but got CUDAIntTensor instead (while checking arguments for embedding)

Found some similar problems here, but with no solution.

Hi Pytorch scholar here,
Path:
deep-learning-v2-pytorch/intro-to-pytorch/Part 5 - Inference and Validation (Solution).ipynb

Line:
accuracy += torch.mean(equals.type(torch.FloatTensor))

Question shuld accuracy be accumulative here?
Batch one 2/5 40% accuracy
Batch two 2/5 40% accuracy
Together = 4/10 should be 40% accuracy

Would it make sense to only have
accuracy = torch.mean(equals.type(torch.FloatTensor))

If we want to calculate the overall accuracy, we should tally up separately
records processed so far
correct results so far

I can be totally wrong. Would appreciate a quick explanation if this is working as expected

Add import numpy as np at the beggining, so you don't have an error later in the section "Numpy to Torch and back" when you try using np.random.rand().

I think it could be helpful for students to see an example of a working network that can classify correctly from the training solution that is given in this section. For those who may have tried but were unable to get their code to work, might be able to learn what they may have done wrong. I was disappointed when I got to this section and no solution was given for this part and all that was said was this is hard and you probably wont get it, but that isn't very helpful for those trying to learn.

image = image.transpose(1,2,0)
image = image * np.array((0.229, 0.224, 0.225)) + np.array((0.485, 0.456, 0.406))
image = image.clip(0, 1)

Section:
TODO: Make dummy variables for rank
one_hot_data = pd.get_dummies(data, columns=['rank'])

TODO: Drop the previous rank column
We don't need drop or use drop_first=True in dummy function

If use drop_first=True for avoid linearity We need hint like: "use drop_first"

Hi,

For the notebook "Part 3 - Training Neural Networks (Solution).ipynb" in Intro to Pytorch, in the 3rd code cell (I pasted it below), I see that we use the nn.LogSoftmax() layer at the end, so I think at the forward pass step, logits = model(images), the name of the output is not logits, I think we should use another name, like just "output" or "logsoftmax_output".

model = nn.Sequential(nn.Linear(784, 128),
                      nn.ReLU(),
                      nn.Linear(128, 64),
                      nn.ReLU(),
                      nn.Linear(64, 10),
                      **nn.LogSoftmax(dim=1)**)

# Define the loss
criterion = nn.NLLLoss()

# Get our data
images, labels = next(iter(trainloader))
# Flatten images
images = images.view(images.shape[0], -1)

**# Forward pass, get our logits
logits = model(images)
# Calculate the loss with the logits and the labels
loss = criterion(logits, labels) ```**

In the test set, there are:

len(testloader.dataset)) = int(10000)

examples.

The with torch.no_grad(): loop runs 157 times (10_000 / 64 examples per minibatch == 156.25)

also, len(testloader)) = int(157)

The issue is with the line:

"Test Accuracy: {:.3f}".format(accuracy/len(testloader)))

This should be dividing by 156.25 not 157.

Make use of the already downloaded image MNIST dataset from part-2 instead of downloading it again to ~/.pytorch/MNIST_data/

the following lines here, here and here causes the following type of error:

Traceback (most recent call last):

  File "/opt/conda/lib/python3.6/site-packages/IPython/core/interactiveshell.py", line 2963, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)

  File "<ipython-input-5-9be11ce52ce0>", line 1, in <module>
    import problem_unittests as tests

  File "/notebooks/project-tv-script-generation/problem_unittests.py", line 65
    assert not (int_to_vocab_word_set - vocab_to_int_word_set),\
    ^
IndentationError: unexpected indent

I can open a PR if you want, but I wasn't sure if it was intentional or not. Let me know. I'd be more than happy to open a PR!

You can access the weight and bias tensors once the network once it's create at net.hidden.weight and net.hidden.bias.

it should be or similar to the sentence below right?

You can access the weight and bias tensors once the network is created at net.hidden.weight and net.hidden.bias.`

What is implemented there is a single-layer NN (input layer + one output node) and this should be stated clearly (i.e. change the "Training the 2-layer Neural Network").

Couple of other things:

1. The function implementing the cross-entropy loss error_formula is actually never used, although it appears in the train_nn function, line error = error_formula(y, output) but the error is unused.

2. There is no bias used in the training, only weights.

3. The error function used is the squared error function, loss = np.mean((out - targets) ** 2) and the error_term is computed as (y - output)*sigmoid_prime(x), not the cross-entropy error as one would expect (see the previous lessons and the error_formula)

4. In the plot_points change the s in list comprehensions to some other variable name because it can be confusing for some students since s = 25 is the marker size:
   plt.scatter([s[0][0] for s in rejected], [s[0][1] for s in rejected], s = 25, color = 'red', edgecolor = 'k')

And a general question: what's the point in using single-layer perceptron for a highly non-classical problem, after introducing the multilayer neural networks and backpropagation? I think it would be better to implement the code for a NN with 1 hidden layer to demonstrate in practice what was learned in previous lesson.

i used solve a problem by using sequential method in nn module but it return an error of size missmatch .I paste my code to here with error

model = nn.Sequential(nn.Linear(784, 256),
nn.ReLU(),
nn.Linear(256, 128),
nn.ReLU(),
nn.Linear(128, 64),
nn.ReLU(),
nn.Linear(64, 10),
nn.LogSoftmax(dim=1))

criterion = nn.NLLLoss()
optimizer = optim.Adam(model.parameters(), lr=0.001)

TODO: Train the network here

epochs = 5

for e in range(epochs):
running_loss = 0
for images, labels in trainloader:
# Flatten MNIST images into a 784 long vector
imagees = images.view(1,-1)

    log_ps = model(images)
    loss = criterion(log_ps, labels)
    
    optimizer.zero_grad()
    loss.backward()
    optimizer.step()
    
    running_loss += loss.item()
else:
    print("Training loss:",running_loss/len(trainloader))

when i run this code at that time it prodce error like :
size mismatch, m1: [1792 x 28], m2: [784 x 256] at c:\programdata\miniconda3\conda-bld\pytorch_1532501501514\work\aten\src\th\generic/THTensorMath.cpp:2070

I am a bit confused. Up to this point instructor was preparing us for multi-layer NN and how to propagate an error up etc. And last section even have label "Training the 2-layer Neural Network"
But from what I see it is simple perceptron with no layers, but only input x and output. Please correct me if I am wrong.

In the GradientDescent.ipynb the weights are updated after each training data example, whereas they should be updated at the end of each epoch, i.e. after processing all the training data set (batch gradient descent algorithm). So the given code actually implements the stochastic gradient descent.
This is inconsistent with the StudentsAddmissions.ipynb where the weights are updated at the end of each epoch (as they should be if we are using GD).

The pre-written code which is used to draw sample images from trainloader (also testloader) draws sample images foggy. Since the reference images in the cell which begins with the line "Your transformed images should look something like this." are nice I thought that I did something wrong.

In the solution notebook, helper.imshow is used with the normalize=False.

Changing helper.imshow(images[ii], ax=ax) to helper.imshow(images[ii], ax=ax, normalize=False)can be helpful.

I think, the thing the instructor actually wants to do with the last 2 codes, is to save the first model with its network architecture. But since he defines right before another model variable with new network architecture, he saves and loads a model which is not trained yet at all.

So the last 3rd code cell which was created to throw an error should be for example like this instead;

# Try this
model_1 = fc_model.Network(784, 10, [400, 200, 100])
# This will throw an error because the tensor sizes are wrong!
model_1.load_state_dict(state_dict)

So that the next 2 code cells (the last 2 code cells) then will save exactly the trained model with its network architecture, not an un-trained new model.

checkpoint = {'input_size': 784,
              'output_size': 10,
              'hidden_layers': [each.out_features for each in model.hidden_layers],
              'state_dict': model.state_dict()}

torch.save(checkpoint, 'checkpoint.pth')

This issue makes the topic somewhat confusing.

Hi, in the "Transfer_Learning_Solution.ipynb", in the training section part, when we update the training loss, there's a line code as follows:

update training loss
train_loss += loss.item()

While in the other 2 notebooks, which are "cifar10_cnn_solution.ipynb" and "mnist_mlp_solution.ipynb", the updating training loss is as follows:

update training loss
train_loss += loss.item()*data.size(0)

Why is there a difference here? Which one is correct: with data.size(0) or without it?
Thank you.

In PyTorch v0.4.1, tensor printing now includes requires_grad and grad_fn information. Therefore, we don't need to print(y.grad_fn) any more.

Literally every person in the course has had issues with loading/working with the helper.py module because the one provided here conflicts with python's built-in helper module.
The module should be renamed to something like helper_mod.py and the corresponding imports renamed in all notebooks to avoid this conflict.

In second code cell Exercises Notebook you assign variable with name 'transforms'. It's overwrite this variable, which contain import from torchvision and can produce Exception after second execution code cell. You have a solution for this bug in Solution Notebook.

This link shows as broken in my jupyter notebook (version 5.7.0):
[uniform distribution](https://en.wikipedia.org/wiki/Uniform_distribution_(continuous%29)
However, the link displays fine in the online notebook viewer http://nbviewer.jupyter.org/.

Proposed fixed link:
[uniform distribution](https://en.wikipedia.org/wiki/Uniform_distribution_(continuous))

It also shows up broken when typed here:
original:
[uniform distribution](https://en.wikipedia.org/wiki/Uniform_distribution_(continuous%29)
fixed:
uniform distribution

Occurrences:

"A [uniform distribution](https://en.wikipedia.org/wiki/Uniform_distribution_(continuous%29) has the equal probability of picking any number from a set of numbers. We'll be picking from a continuous distribution, so the chance of picking the same number is low. We'll use NumPy's `np.random.uniform` function to pick random numbers from a uniform distribution.\n",

"A [uniform distribution](https://en.wikipedia.org/wiki/Uniform_distribution_(continuous%29) has the equal probability of picking any number from a set of numbers. We'll be picking from a continuous distribution, so the chance of picking the same number is low. We'll use NumPy's `np.random.uniform` function to pick random numbers from a uniform distribution.\n",