the setting about torch.autograd.grad function about maml-pytorch HOT 8 CLOSED

Hello!

Is there a consensus on this? Unless I am missing something, I also think you might need create_graph = True in order to propagate your gradients correctly through the inner gradient.

from maml-pytorch.

I also think it needs create_graph = True.
grad = torch.autograd.grad(loss, self.net.parameters())
My understanding is that the line above will give you grad.requires_grad = False. Thus, there is no higher order gradients available. It's a first order MAML.
grad = torch.autograd.grad(loss, self.net.parameters(), create_graph = True)
The line above will give you grad.requires_grad = True.

from maml-pytorch.

Generally speaking, you should set create_graph for 2nd order gradient.
However, the implementaton in this repo. is the 2nd order version.

from maml-pytorch.

I use pytorch1.0.The default setting create_graph is False.
And I checked the grad on Meta.py line 86 and line116.
grad = torch.autograd.grad(loss, fast_weights)
The require_grads about grad is False.
I am not sure the implementaton in this repo. is the 2nd order version.

from maml-pytorch.

@lynlynlyn IT's 2nd order implementation.
I think you should digest the paper to understand how to derive 2nd order gradient.
It's not simply set create_graph=True.
Sorry.

from maml-pytorch.

if the inner loop gradients's require_grads=False,where is the 2nd order gradient?

from maml-pytorch.

@Deng-Y well, I have not worked on this project a long time. sorry for misleading information. Have you tried setting create_grape=True?

from maml-pytorch.

@dragen1860 I only try it on the other project. Not sure if it will improve the performance on this one.

from maml-pytorch.