dsgiitr / d2l-pytorch Goto Github PK

in CH_4_Data_manipulation, we use

We can transform the result into a scalar in Python using the asscalar function of numpy. In the following example, the $\ell_2$ norm of x yields a single element tensor. The final result is transformed into a scalar.
In[20]: np.asscalar(x.norm())
Out[20]: 22.494443893432617

Why don't just use x.norm().item() to get the scalar, which seems much easier? Are their any differences?

Hi Team,

In Ch06_Multilayer_Perceptrons/Multilayer_Perceptron.ipynb, I found that the y axis label for the gradient of relu() is mistakenly written as grad of sigmoid.

Can I find the answer of the practising problems in somewhere?

I would cover the sections 12.3, 12.6 and 12.10 in the upcoming PR if nobody is working on it.

Chapter 10.6 Concise Implementation of Recurrent Neural Networks was missing.

Hope to see this part, and thanks for your open source code.

This is the original code

def resnet_block(input_channels, num_channels, num_residuals, first_block=False):
  blk = []
  for i in range(num_residuals):
    if i == 0 and not first_block:
      blk.append(Residual(input_channels, num_channels, use_1x1conv=True, strides=2))
    else:
      blk.append(Residual(num_channels, num_channels))
  return blk

I tried running the code in colab but there was a channel error. so I added the last line of code

in_channels = out_channels

And the new code looks like this .and it runs ok.

def resnet_block(in_channels,out_channels,num_residuals,first_block = False):
    blk = []
    for i in range(num_residuals):
        if i == 0 or not first_block:
            blk.append(Residual(in_channels,out_channels,use_1x1conv= True,strides = 2))
        else:
            blk.append(Residual(out_channels,out_channels))
        in_channels = out_channels
    return blk

We can implement functions like tensor rotation, scaling, translation, general and channel-wise etc,
which are currently not possible with direct use of pytorch library as there are no such direct functions implemented, one idea was to convert tensor to PIL image and apply transformations and then revert back to tensor, the idea seems legit but there seems to be a bug in the library that doesn't return the correct result after conversion back to tensor, so such custom functions can be very helpful.

Hi, I was following your tutorial on https://d2l.ai/chapter_computer-vision/bounding-box.html
How can I save figure with bounding boxes on it?

in Ch09_Modern_Convolutional_Networks/Network_in_Network(NiN).ipynb defines the NiN net model with:

    #Global Average Pooling can be achieved by AdaptiveMaxPool2d with output size = (1,1)
    self.avg1 = nn.AdaptiveMaxPool2d((1,1))

Is it a typo of nn.AdaptiveAvgPool2d or some deeper reasons behind this? Cause using Avg will train a lot slower and get bad train/val accuracy like 0.460 (epoch 5)

Two questions

1. alpha_t is unused below.
2. should we use the following to derive the BCECrossEntropyLoss:

pt = torch.log(p) * target.float() + torch.log(1.0 - p) * (1 - target).float()

class FocalLoss(nn.Module):
    def __init__(self, alpha=0.25, gamma=2, device="cuda:0", eps=1e-10):
        super().__init__()
        self.alpha = alpha
        self.gamma = gamma
        self.device = device
        self.eps = eps

    def forward(self, input, target):
        p = torch.sigmoid(input)
        pt = p * target.float() + (1.0 - p) * (1 - target).float()
        alpha_t = (1.0 - self.alpha) * target.float() + self.alpha * (1 - target).float()
        loss = - 1.0 * torch.pow((1 - pt), self.gamma) * torch.log(pt + self.eps)
        return loss.sum()

Hey, really great work on porting the book's code over to PyTorch.

It's one of the nicest books I read (atleast the few chapters I did) and Zack & team are great writers.

We want to get this port some attention via PyTorch Twitter, and want to make sure this book is ready in a state that you like -- and that we aren't surprising you by pre-announcing.

Let us know

I would like to contribute to this topic with notebook using Pytorch. Kindly assign this to me.

not implement “Image Classification (CIFAR-10) on Kaggle” ？

there is a runtime error when running the 7th code block. But I checked, the device type doesn't conflict each other. What's Wrong?

RuntimeError Traceback (most recent call last)
in
1 state = init_rnn_state(X.shape[0], num_hiddens, ctx)
2 inputs = to_onehot(X.to(ctx), len(vocab))
----> 3 params = get_params()
4 outputs, state_new = rnn(inputs, state, params)
5 len(outputs), outputs[0].shape, state_new[0].shape

in get_params()
9
10 # Hidden layer parameters
---> 11 W_xh = _one((num_inputs, num_hiddens))
12 W_hh = _one((num_hiddens, num_hiddens))
13 b_h = torch.zeros(num_hiddens, device=ctx)

in _one(shape)
6 def get_params():
7 def one(shape):
----> 8 return torch.Tensor(size=shape, device=ctx).normal(std=0.01)
9
10 # Hidden layer parameters

RuntimeError: legacy constructor for device type: cpu was passed device type: cuda, but device type must be: cpu

In Chp14_Computer_Vision Single_Shot_Multibox_Detection.ipynb in the implementation of Define Loss and Evaluation Functions there is a missing .long() on line 61. A TypeError was raised: "Expected Long but got Float"

I solved it by changing class_true_i[0, j] to class_true_i[0, j].long() because class_target is a tensor with dtype long as shown on line 54 class_target = torch.zeros(class_hat_i.shape[0]).long().to(self.device)

Hi authors,

it seems that download_and_preprocess_data() method does not work?

can you check?

Thanks

In Section "Head gradients and the chain rule"
z.backward(head_gradient)
should be modified to
y.backward(head_gradient)

To exactly replicate the book, function d2l.train_ch3() function would be a necessary addition. This function is to be called during training from the implementations in other chapters as well. This would be more convenient than writing the training loop each and every time.
Infact, all training loops required in the book can be added to train.py similar to https://github.com/d2l-ai/d2l-en/blob/8e53fce19c6c744cf4994a896c43e382567e6fbc/d2l/train.py#L1

Replace with : 'cuda:'+str(i)