aioz-ai / miccai21_mmq Goto Github PK

View Code? Open in Web Editor NEW

33.0 33.0 10.0 286 KB

Multiple Meta-model Quantifying for Medical Visual Question Answering (MICCAI 2021)

Home Page: https://blog.ai.aioz.io/research/vqa-mmq/

License: MIT License

Python 98.29% Shell 1.71%

deep-learning medical medical-image-processing question-answering vqa

miccai21_mmq's People

Contributors

Stargazers

Watchers

Forkers

hannibalwanglecter maejav yunyiliu hertera1 victory0622 aioznetwork

miccai21_mmq's Issues

The dataset download links do not work

Hi everyone,

It seems like the dataset download links do not work again.
Could you please help me check it?

Thank you for your great work!

Selection of the best model

Hello, we also encountered the same problem, that is, after training 40 epochs, how do we choose the best model to test the accuracy on the test set? Is it to load all the generated weight models of 40 epochs in order to test the accuracy in the test set? Then choose the highest precision as the best result?Thank you very much for your reply.

the valset of rad

Hello, I recently read the MMQ paper and code, there is a question, that is, the RAD dataset does not have a validation set, then I want to ask, what do we rely on to find the best accuracy model, and then use the test set to test the final accuracy? Thank you very much for your answer.

Model accuracy

hello author，How much is the experimental error made in accordance with your parameters?

the detai link in "README" to download VQA-RAD dataset in data/vqarad_maml

Can you provide the detail link in "Readme" to download the VQA-RAD dataset in "data/vqarad_maml"? Thanks a lot!!!

Could you please tell me how to get the embed_tfidf_weights.pkl

cannot access the link for downloading the pretrained models and datasets

hello, I cannot get access to the links for downloading the pretrained models and the datasets. Does the links work well? Thanks!

The problem of the accuracy

Hello! I have tried to run the command line "sh run_vqa_PathVQA.sh" and then run "sh run_test_PathVQA.sh", but the total accuracy is 46.6%. I have tried it for many times, but the total accuracy I get is always around 46.6%.Can you please tell me how can I get the total accuracy on 'MMQ + MEVF' which is 49.0%?

Discuss on the reproducibility

Hi authors,

I am trying to reproduce the results of your method and the baselines.
It looks good since the results I got are very close to your reported ones, in multiple random seeds.

However, my observations are:

With the same random seed and same GPU, different runs will give different losses, valid results, and test results (around 1-2%)
With the same GPU and trained model, testing in different times will give different results sometimes.
Easy to see that the problems above will happen when we use different GPUs too.
(different GPUs but the same GPU model and configurations)

Personally, I think this issue did occur from the baseline's implementation...

Could you please guide me on how to fix this issue?

Thank you in advance!

cannot access the link for downloading the pretrained models and datasets

hello, I cannot get access to the links for downloading the pretrained models and the datasets,too.

the problem of accuracy

Hello, I use the VQA-RAD dataset to train according to the instructions provided by readme, and finally train forty epochs to appear very poor, open only 28 accuracy, close only 70 accuracy, want to ask, is it too little for me to train 40 epoch by default, or may there be problems somewhere, thank you for your help

could you give the code for extract ae and maml featrue, we want to extract feature on own dataset

could you help me

When i run $ sh run_pathVQA.sh
Prompts for this error
FileNotFoundError: [Errno 2] No such file or directory: 'saved_models/maml84_miccai2021_optimization_newmethod_6way_5shot_t4/model_epoch0.pth'
and UnboundLocalError: local variable 'i' referenced before assignment
Thank you very much for you patience

A question about test code

Dear author,
I found that in source code test.py, line 180, which inside function get_result_RAD(), you filter questions whose p_type[0] is not 'freedom'. I wonder what's the difference between 'freedom' and 'para', and why you choose to discard those non-freedom samples.
Sincerely

Cannot run script : No such file or directory

Hi, I installed all the requirements and run the sh file, but it reports an error, here is the information :

sh run_pathVQA.sh
Namespace(data='data/pathvqa_maml/', epoch=10000, imgc=3, imgsz=84, k_qry=15, k_spt=5, meta_lr=0.001, n_way=6, output='saved_models', t_dst=0, task_num=4, update_lr=0.01, update_step=5, update_step_test=10)
Meta(
  (net): Learner(
    conv2d:(ch_in:3, ch_out:32, k:3x3, stride:1, padding:0)
    relu:(True,)
    bn:(32,)
    max_pool2d:(k:2, stride:2, padding:0)
    conv2d:(ch_in:32, ch_out:32, k:3x3, stride:1, padding:0)
    relu:(True,)
    bn:(32,)
    max_pool2d:(k:2, stride:2, padding:0)
    conv2d:(ch_in:32, ch_out:32, k:3x3, stride:1, padding:0)
    relu:(True,)
    bn:(32,)
    max_pool2d:(k:2, stride:2, padding:0)
    conv2d:(ch_in:32, ch_out:32, k:3x3, stride:1, padding:0)
    relu:(True,)
    bn:(32,)
    max_pool2d:(k:2, stride:1, padding:0)
    flatten:()
    linear:(in:800, out:6)
    
    (vars): ParameterList(
        (0): Parameter containing: [torch.cuda.FloatTensor of size 32x3x3x3 (GPU 0)]
        (1): Parameter containing: [torch.cuda.FloatTensor of size 32 (GPU 0)]
        (2): Parameter containing: [torch.cuda.FloatTensor of size 32 (GPU 0)]
        (3): Parameter containing: [torch.cuda.FloatTensor of size 32 (GPU 0)]
        (4): Parameter containing: [torch.cuda.FloatTensor of size 32x32x3x3 (GPU 0)]
        (5): Parameter containing: [torch.cuda.FloatTensor of size 32 (GPU 0)]
        (6): Parameter containing: [torch.cuda.FloatTensor of size 32 (GPU 0)]
        (7): Parameter containing: [torch.cuda.FloatTensor of size 32 (GPU 0)]
        (8): Parameter containing: [torch.cuda.FloatTensor of size 32x32x3x3 (GPU 0)]
        (9): Parameter containing: [torch.cuda.FloatTensor of size 32 (GPU 0)]
        (10): Parameter containing: [torch.cuda.FloatTensor of size 32 (GPU 0)]
        (11): Parameter containing: [torch.cuda.FloatTensor of size 32 (GPU 0)]
        (12): Parameter containing: [torch.cuda.FloatTensor of size 32x32x3x3 (GPU 0)]
        (13): Parameter containing: [torch.cuda.FloatTensor of size 32 (GPU 0)]
        (14): Parameter containing: [torch.cuda.FloatTensor of size 32 (GPU 0)]
        (15): Parameter containing: [torch.cuda.FloatTensor of size 32 (GPU 0)]
        (16): Parameter containing: [torch.cuda.FloatTensor of size 6x800 (GPU 0)]
        (17): Parameter containing: [torch.cuda.FloatTensor of size 6 (GPU 0)]
    )
    (vars_bn): ParameterList(
        (0): Parameter containing: [torch.cuda.FloatTensor of size 32 (GPU 0)]
        (1): Parameter containing: [torch.cuda.FloatTensor of size 32 (GPU 0)]
        (2): Parameter containing: [torch.cuda.FloatTensor of size 32 (GPU 0)]
        (3): Parameter containing: [torch.cuda.FloatTensor of size 32 (GPU 0)]
        (4): Parameter containing: [torch.cuda.FloatTensor of size 32 (GPU 0)]
        (5): Parameter containing: [torch.cuda.FloatTensor of size 32 (GPU 0)]
        (6): Parameter containing: [torch.cuda.FloatTensor of size 32 (GPU 0)]
        (7): Parameter containing: [torch.cuda.FloatTensor of size 32 (GPU 0)]
    )
  )
)
Total trainable tensors: 33702
shuffle DB :train, b:10000, 6-way, 5-shot, 15-query, resize:84
--loading data from: data/pathvqa_maml/t0/train
Traceback (most recent call last):
  File "pathVQA_maml_train.py", line 133, in <module>
    main(args)
  File "pathVQA_maml_train.py", line 68, in main
    batchsz=10000, resize=args.imgsz, t = args.t_dst)
  File "/home/wanghao/Code/VQA/MICCAI21_MMQ/mmq_maml/pathVQA_maml.py", line 72, in __init__
    self.img2label['x'] = i + 1 + self.startidx
UnboundLocalError: local variable 'i' referenced before assignment
Traceback (most recent call last):
  File "pathVQA_maml_half.py", line 224, in <module>
    main(args)
  File "pathVQA_maml_half.py", line 136, in main
    maml.load_state_dict(torch.load(model_path))
  File "/home/zhhliu/anaconda3/envs/MMQ/lib/python3.6/site-packages/torch/serialization.py", line 356, in load
    f = open(f, 'rb')
FileNotFoundError: [Errno 2] No such file or directory: 'saved_models/maml84_miccai2021_optimization_newmethod_6way_5shot_t0/model_epoch0.pth'
Traceback (most recent call last):
  File "pathVQA_maml_train.py", line 133, in <module>
    main(args)
  File "pathVQA_maml_train.py", line 68, in main
    batchsz=10000, resize=args.imgsz, t = args.t_dst)
  File "/home/wanghao/Code/VQA/MICCAI21_MMQ/mmq_maml/pathVQA_maml.py", line 72, in __init__
    self.img2label['x'] = i + 1 + self.startidx
UnboundLocalError: local variable 'i' referenced before assignment
Traceback (most recent call last):
  File "pathVQA_maml_half.py", line 224, in <module>
    main(args)
  File "pathVQA_maml_half.py", line 136, in main
    maml.load_state_dict(torch.load(model_path))
  File "/home/zhhliu/anaconda3/envs/MMQ/lib/python3.6/site-packages/torch/serialization.py", line 356, in load
    f = open(f, 'rb')
FileNotFoundError: [Errno 2] No such file or directory: 'saved_models/maml84_miccai2021_optimization_newmethod_6way_5shot_t1/model_epoch0.pth'
Traceback (most recent call last):
  File "pathVQA_maml_train.py", line 133, in <module>
    main(args)
  File "pathVQA_maml_train.py", line 68, in main
    batchsz=10000, resize=args.imgsz, t = args.t_dst)
  File "/home/wanghao/Code/VQA/MICCAI21_MMQ/mmq_maml/pathVQA_maml.py", line 72, in __init__
    self.img2label['x'] = i + 1 + self.startidx
UnboundLocalError: local variable 'i' referenced before assignment
Traceback (most recent call last):
  File "pathVQA_maml_half.py", line 224, in <module>
    main(args)
  File "pathVQA_maml_half.py", line 136, in main
    maml.load_state_dict(torch.load(model_path))
  File "/home/zhhliu/anaconda3/envs/MMQ/lib/python3.6/site-packages/torch/serialization.py", line 356, in load
    f = open(f, 'rb')
FileNotFoundError: [Errno 2] No such file or directory: 'saved_models/maml84_miccai2021_optimization_newmethod_6way_5shot_t2/model_epoch0.pth'
Traceback (most recent call last):
  File "pathVQA_maml_train.py", line 133, in <module>
    main(args)
  File "pathVQA_maml_train.py", line 68, in main
    batchsz=10000, resize=args.imgsz, t = args.t_dst)
  File "/home/wanghao/Code/VQA/MICCAI21_MMQ/mmq_maml/pathVQA_maml.py", line 72, in __init__
    self.img2label['x'] = i + 1 + self.startidx
UnboundLocalError: local variable 'i' referenced before assignment
Traceback (most recent call last):
  File "pathVQA_maml_half.py", line 224, in <module>
    main(args)
  File "pathVQA_maml_half.py", line 136, in main
    maml.load_state_dict(torch.load(model_path))
  File "/home/zhhliu/anaconda3/envs/MMQ/lib/python3.6/site-packages/torch/serialization.py", line 356, in load
    f = open(f, 'rb')
FileNotFoundError: [Errno 2] No such file or directory: 'saved_models/maml84_miccai2021_optimization_newmethod_6way_5shot_t3/model_epoch0.pth'
Traceback (most recent call last):
  File "pathVQA_maml_train.py", line 133, in <module>
    main(args)
  File "pathVQA_maml_train.py", line 68, in main
    batchsz=10000, resize=args.imgsz, t = args.t_dst)
  File "/home/wanghao/Code/VQA/MICCAI21_MMQ/mmq_maml/pathVQA_maml.py", line 72, in __init__
    self.img2label['x'] = i + 1 + self.startidx
UnboundLocalError: local variable 'i' referenced before assignment
Traceback (most recent call last):
  File "pathVQA_maml_half.py", line 224, in <module>
    main(args)
  File "pathVQA_maml_half.py", line 136, in main
    maml.load_state_dict(torch.load(model_path))
  File "/home/zhhliu/anaconda3/envs/MMQ/lib/python3.6/site-packages/torch/serialization.py", line 356, in load
    f = open(f, 'rb')
FileNotFoundError: [Errno 2] No such file or directory: 'saved_models/maml84_miccai2021_optimization_newmethod_6way_5shot_t4/model_epoch0.pth'
Traceback (most recent call last):
  File "pathVQA_maml_train.py", line 133, in <module>
    main(args)
  File "pathVQA_maml_train.py", line 68, in main
    batchsz=10000, resize=args.imgsz, t = args.t_dst)
  File "/home/wanghao/Code/VQA/MICCAI21_MMQ/mmq_maml/pathVQA_maml.py", line 72, in __init__
    self.img2label['x'] = i + 1 + self.startidx
UnboundLocalError: local variable 'i' referenced before assignment
Traceback (most recent call last):
  File "pathVQA_maml_fuse.py", line 175, in <module>
    main(args)
  File "pathVQA_maml_fuse.py", line 137, in main
    batchsz=600, resize=args.imgsz)
  File "/home/wanghao/Code/VQA/MICCAI21_MMQ/mmq_maml/pathVQA_maml.py", line 72, in __init__
    self.img2label['x'] = i + 1 + self.startidx
UnboundLocalError: local variable 'i' referenced before assignment
run_pathVQA.sh: line 29: Namespace(data='data/pathvqa_maml/',: No such file or directory

code for generating the files in the data_PathVQA/ directory

Hello, it is an excellent work, and I wonder if you could share the code for generating the files such as trainset.json, testset.json pytorch_images84.pkl, pytorch_images128_ae.pkl in data_pathVQA directory with the original data. Thanks!

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.