The dsw from vinairesearch

With DSWD as a loss, should backprop go through the function f?

Hi,

Thank you for sharing your code. I am wondering if this line

Line 126 in d76ffb1

projections = f(pro)

should be replaced by

with torch.inference_mode():
    projections = f(pro)

When updating a generative model that uses DSWD as a loss, do we need the computational graph of f that comes attached to projections? i.e., can projections be treated as a constant when backpropagating the DSWD loss through a generative model?

Pytorch Version

Hi! Thank you for releasing the code for such an interesting project.

I noticed your code expects an older version of pytorch. Would you consider a version that works with a newer pytorch (e.g., on a different branch)?

CUDA out of memory

Thank you for sharing the codes of this interesting tool.
I get the error below when running the code in Jupyter. However, this error does not appear when running on google colab.
I created a new enviornment with Python 3.8.10 and installed torch==1.4.0 torchvision==0.5.0 numpy tqdm POT scikit-image.
Any tips to tackle this error please? :)
run mnist.py --datadir=./ --outdir=./result --batch-size=1 --seed=16 --p=2 --lr=0.05 --dataset=MNIST --model-type=DSWD --latent-size=32 --num-workers=0

RuntimeError                              Traceback (most recent call last)
~\Sliced Gan\DSW-master\DSW-master\mnist.py in <module>
    327 
    328 if __name__ == "__main__":
--> 329     main()

~\Sliced Gan\DSW-master\DSW-master\mnist.py in main()
    281                     fake = model.decoder(fixednoise_wd)
    282                     wd_list.append(compute_true_Wasserstein(data.to("cpu"), fake.to("cpu")))
--> 283                     swd_list.append(sliced_wasserstein_distance(data, fake, 10000).item())
    284                     print("Iter:" + str(ite) + " WD: " + str(wd_list[-1]))
    285                     np.savetxt(model_dir + "/wd.csv", wd_list, delimiter=",")

~\Sliced Gan\DSW-master\DSW-master\utils.py in sliced_wasserstein_distance(first_samples, second_samples, num_projections, p, device)
     17     wasserstein_distance = torch.abs(
     18         (
---> 19             torch.sort(first_projections.transpose(0, 1), dim=1)[0]
     20             - torch.sort(second_projections.transpose(0, 1), dim=1)[0]
     21         )

RuntimeError: CUDA out of memory. Tried to allocate 1.49 GiB (GPU 0; 4.00 GiB total capacity; 2.00 GiB already allocated; 935.90 MiB free; 2.02 GiB reserved in total by PyTorch)

vinairesearch / dsw Goto Github PK

dsw's People

Contributors

Stargazers

Watchers

Forkers

dsw's Issues

With DSWD as a loss, should backprop go through the function f?

Pytorch Version

CUDA out of memory

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent