cuthbertcai / pytorch_dann Goto Github PK

Hi there, thanks for the great repo! I would like to point out that I think the dropout function should be initialized at init in the in Class_classifier, so that. Your implementation uses dropout inplace.

logits = self.fc2(F.dropout(logits))

By adding self.dropout = nn.Dropout() during the initialization, and replace this term with

logits = self.fc2(logits)
logits = self.dropout(logits)

I was able to obtain a 2~3% performance gain on the tasks using the same model checkpoint and inputs.
Could you help me verify if my understanding is correct?

When I run the main.py, the error comes :RuntimeError: output with shape [1, 28, 28] doesn't match the broadcast shape [3, 28, 28]

I tried to run the code on py2.7 but it keeps giving me an error -
../models/models.py", line 68, in forward
input = GradReverse.grad_reverse(input, constant)
TypeError: unbound method grad_reverse() must be called with GradReverse instance as first argument (got Variable instance instead)
I tried with some more combinations of versions (py-3.5, torch 0.4.1, etc.), all gave some or the other error.
I suspect it is due to some version issue of some package. It will be very much helpful if you can upload environment.yml file or version details in a req.txt file.

I found you have set the gamma = -10 in params.py. That means the lambda is always minus, but you have use grad_output = grad_output.neg() in GRL in the code models.py, that means the grad is always positive, which it is different from what the article said.

There is a minor error in the train and test dataloaders for the MNIST dataset. The transform for normalizing data values receives mean and standard deviation for three channels whereas the MNIST images only have a single channel. We can just fix this by changing the values in the transforms such as:

if dataset == 'MNIST':
        transform = transforms.Compose([
                                        transforms.ToTensor(),
                                        transforms.Normalize(mean= params.dataset_mean[0], std= params.dataset_std[0])
                                        ])

This way only one channel is transformed as it should.

Hi,
The result of my code has huge difference with yours. At the epoch 99, my result is:

Source Accuracy: 9839/10000 (98.0000%)
Target Accuracy: 5958/9001 (66.0000%)
Domain Accuracy: 10646/19001 (56.0000%)

May I ask how much epoch did you run to get that result please?
Thank you in advance.

Hi,

I am reading your codes in the file with the name train.py. I think the parameter for the trade-off of class loss and domain loss is missing. Would you please have a look?

Thank you very much.

Hi,

I'm not sure if this is an issue, or that's my misunderstanding. You calculated loss as:

loss = class_loss + params.theta * domain_loss

shouldn't it be:

loss = class_loss - params.theta * domain_loss

according to the formula (9) of the paper?

I've been struggling too much days with this error, maybe it's really simple but i can't figure out the reason yet. The error is the following:

Traceback (most recent call last):
File "main.py", line 205, in
main(parse_arguments(sys.argv[1:]))
File "main.py", line 169, in main
src_train_dataloader, tgt_train_dataloader, optimizer, epoch)
File "/Volumes/Elements/DANN/pytorch_DANN-master/train/train.py", line 36, in train
for batch_idx, (sdata, tdata) in enumerate(zip(source_dataloader, target_dataloader)):
File "/Volumes/Elements/Apps/anaconda3/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 615, in next
batch = self.collate_fn([self.dataset[i] for i in indices])
File "/Volumes/Elements/Apps/anaconda3/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 615, in
batch = self.collate_fn([self.dataset[i] for i in indices])
File "/Volumes/Elements/Apps/anaconda3/lib/python3.7/site-packages/torchvision/datasets/mnist.py", line 95, in getitem
img = self.transform(img)
File "/Volumes/Elements/Apps/anaconda3/lib/python3.7/site-packages/torchvision/transforms/transforms.py", line 60, in call
img = t(img)
File "/Volumes/Elements/Apps/anaconda3/lib/python3.7/site-packages/torchvision/transforms/transforms.py", line 163, in call
return F.normalize(tensor, self.mean, self.std, self.inplace)
File "/Volumes/Elements/Apps/anaconda3/lib/python3.7/site-packages/torchvision/transforms/functional.py", line 208, in normalize
tensor.sub_(mean[:, None, None]).div_(std[:, None, None])
IndexError: too many indices for tensor of dimension 0

I'd appreciate a lot your help.

Hi @tengerye @CuthbertCai! Thank you for sharing the code.

I was wondering if this particular domain adaptation technique can be used in a regression problem. Let's say if I want to use this domain adaptation technique for a regression problem where the loss function for the main task is MAE or MSE. I am a bit confused about how to actually backpropagate the loss in such a case. Since in the classification problem, the total loss is a summation (according to the code) of the class loss and domain loss and as both of them are negative log-likelihood loss there is no issue while adding them. However, if the loss for the actual task and the loss for the domain classification task are different what should we do in that case? Your insight would be really helpful! Thanks!