quick explanation of pytorch lstm about pytorch-tutorial HOT 1 CLOSED

def forward(self, x):
    # Set initial states 
    h0 = Variable(torch.zeros(self.num_layers, x.size(0), self.hidden_size)) 
    c0 = Variable(torch.zeros(self.num_layers, x.size(0), self.hidden_size))
        
    # Forward propagate RNN
    out, (h_last, c_last) = self.lstm(x, (h0, c0))    # same as self.lstm(x). Default is zero initialization.
        
    # Decode hidden state of last time step
    out = self.fc(out[:, -1, :])             # same as out = h_last
    return out

@sinhasam You should note that images is a 3D tensor of shape (batch_size, seq_length, input_size). In out, (h_last, c_last) = lstm(x, (h0, c0)), out is a tensor of shape (batch_size, seq_length, hidden_size) and h_last and c_last are tensors of shape (batch_size, hidden_size) which indicate the last hidden and cell state of the lstm (Full time step forward propagation, not one-step). You can alse see here to understand the mechanism of nn.LSTM.

In addition, in many-to-one case, we need only last hidden state of the lstm. For this, we can use out[:, -1, :] or h_last. In our case (MNIST), each sequence in x has a fixed length of 28 so output[:, -1, :] and h_last are exactly same. For variable length many-to-one, you can use pack_padded_sequence and h_last. Please, see here for the details.

from pytorch-tutorial.