haha-dl / mldg Goto Github PK

I am wondering what is the best practice to apply your method to bigger architectures, e.g. ResNet-50. Should I replace all conv layers in the resnet model?

Thanks in advance

"os.mkdir" can not create multi tier directory.

After I read the paper, one question remains for me, you wrote that the domain S is splited into meta-train domains and meta-test domains.
For example, the domain can be the style of images, image A has two styles: cartoon and realistic image.
Then in the Alg. 1, the meta train loss is calculated on the A's cartoon style, and meta val loss is calculated on the A's realistic style?
Or the two loss are computed on completed different images' styles: A's cartoon style for meta train loss, and image B's realistic style for meta val loss?
yours sincerely.
Thank you!

I see you compared with DANN, etc. can you release code for that?

Thanks for your implementation.

I have run run_mldg.sh two times and check the result in file run_1 and run_2.

In each of the logs_mldg_(0-3), I check the heldout_test_log.txt and flag_log.txt.

I got following results:
run_1:
logs_mldg_0: unseen_index=0 heldout_test_log=0.70849
logs_mldg_1: unseen_index=1 heldout_test_log=0.47440
logs_mldg_2: unseen_index=2 heldout_test_log=0.95449
logs_mldg_3: unseen_index=3 heldout_test_log=0.39246
run_2:
logs_mldg_0: unseen_index=0 heldout_test_log=0.72363
logs_mldg_1: unseen_index=1 heldout_test_log=0.48677
logs_mldg_2: unseen_index=2 heldout_test_log=0.95748
logs_mldg_3: unseen_index=3 heldout_test_log=0.40697
The average is just around 0.63, and each result is much different from the result reported in paper. So it is hard to believe this is the accuracy regarding each unseen domain(A C P S).

I don't change any parameters in run_mldg.sh and main_mldg.py
Did I do something wrong or I misunderstood something?

Thank you very much.

@Yash-Vardhan-Sharma
Thanks for your interests in our paper.

This was originally for demo purpose and for people to easily run the algorithm, so the backbone is just a shallow MLP trained with the pre-extracted ResNet-18 feature. That is why the result is different from the reported.

Ok, I will update the repo to replace the backbone as the one used in the paper to remove confusion.

Originally posted by @dali-dl in #2 (comment)

Could you please upload the version with new backbone, thanks.

Hi, thank you for this fascinating work and providing a demo of MLDG.

Two quick questions:

1. Did you use the first-order approximation in the MLP version of MLDG. The codes in ops.py look like an operation of the first-order approximation.

`

    if not stop_gradient:
        grad_weight = autograd.grad(meta_loss, weight, create_graph=True)[0]

        if bias is not None:
            grad_bias = autograd.grad(meta_loss, bias, create_graph=True)[0]
            bias_adapt = bias - grad_bias * meta_step_size
        else:
            bias_adapt = bias

    else:
        grad_weight = Variable(autograd.grad(meta_loss, weight, create_graph=True)[0].data, requires_grad=False)

        if bias is not None:
            grad_bias = Variable(autograd.grad(meta_loss, bias, create_graph=True)[0].data, requires_grad=False)
            bias_adapt = bias - grad_bias * meta_step_size
        else:
            bias_adapt = bias

    return F.linear(inputs,
                    weight - grad_weight * meta_step_size,
                    bias_adapt)
else:
    return F.linear(inputs, weight, bias)`

2. I am also wondering the meaning of the parameter "--stop_gradient". What would happen when we set it ture?