sara-nl / 3d-vq-vae-2 Goto Github PK

@robogast

Happy to report I was able to train a VQ-VAE using a dataset. Very cool to see - and kudos for the nice Tensorboard outputs you have in place! 😎

1. Do you have any suggestions or code for randomly sampling from the decoder in a generative fashion?

2. Also, If you have a summary of these files and their purpose, that would be very helpful. I would be happy to do a PR with some comments in the repository if that would be helpful.

Questions on:
calc_ssim_from_checkpoint.py # does this calculate SSIM across the dataset ❓
decode_embeddings.py # Specifications for db_path ❓
extract_embeddings.py # Does this save embedding to disk ❓

Ran successfully:
plot_from_checkpoint.py # plots a forward pass from a random sample ✅
train.py # trains a model ✅

Much appreciated!
-Akshay

Hi,

I'm using your implementation to generate MRIs. I have trained a VQ-VAE to reconstruct 3D MRIs, but I am unsure about which vectors to use for training the PixelCNN for sampling.

I attempted to understand your LMDB implementation, but it would take me a significant amount of time to fully grasp it. I'm not clear on what exactly is being stored in the LMDB database.

Given that the VQ-VAE encoder outputs multiple quantization vectors (one for each encoding block), what should be the specific input for the PixelCNN?

x = torch.randn(4, 3, 128, 128, 64).to('cuda')
decoded, (commitment_loss, quantizations, encoding_idx) = vqvae(x)

I think i'll have to modify the LMDB data module part.

Thank you!

When installing the dependencies, I am getting:

$ conda env create -f environment.yml
Collecting package metadata (repodata.json): done
Solving environment: failed

ResolvePackageNotFound: 
  - torchvision==0.9.0.dev20210122=py38_cu110
  - pytorch==1.8.0.dev20210122=py3.8_cuda11.0.221_cudnn8.0.5_0
  

Any ideas?

Hello,

I wanted to confirm the steps for training a VQ-VAE on radiology data. Thank you for working on such an interesting and important application of VQ-VAE. Our research group is particularly interested in applications to oncologic imaging.

1. Sample data

• Is there sample imaging data in: https://surfdrive.surf.nl/files/index.php/s/xY7bwjrgfnhPCAt?path=%2F or would you suggest we use other radiology datasets?
• Also, if we use our own dataset do you have details on how these should be formatted or structured? (I.e. Dicom files or NIfTI files)

2. Training

• Would you recommend leaving the default parameters in the vqvae/train.py script (aside from the dataset)?
• Any suggestions here would be very much appreciated!