We spend a lot of time collecting and summarizing relevant papers and datasets, where you can find them at https://github.com/Ivygugu/Pulmonary-Embolism-Detection/blob/main/README.md

This code is a pytorch implementation of our paper "PE-MVCNET: MULTI-VIEW AND CROSS-MODAL FUSION NETWORK FOR PULMONARY EMBOLISM PREDICTION".

Our method comprises the Image-only module with an integrated multi-view block, the EMR-only module, and the Cross-modal Attention Fusion (CMAF) module. These modules cooperate to extract comprehensive features that subsequently generate predictions for PE. The figure below shows our proposed network.

We compared to other state-of-the-art methods and our results are better than any other method of comparison. Results prove that the introduced CMAF module adeptly captures the inherent correlations between the two modalities, thereby providing the model with richer information.

• Linux

• Python>=3.7

• NVIDIA GPU (memory>=23G) + CUDA cuDNN

pip install -r requirements.txt

Our model's best checkpoint is provided in this link https://drive.google.com/file/d/1RAhGWlnhfWrR4VFcd0w_a_pskoMam7hd/view?usp=drive_link, and you can use it to easily replicate our results.

The preprocessing of the image data can follow this link https://github.com/marshuang80/PENet In short, using create_hdf5.py to make an hdf5 file. As for the preprocessing of the tabular data, We first conduct dimensionality reduction using LinearSVC on the EMR data, then use TabNet to transform the data into suitable embeddings.

To do the evaluation process, please run the following command :

sh test.sh

If you want to find the best threshold for the result, please run the following command :

python best_threshold.py

If you want to train by yourself, you can run this command :

sh train.sh

The data we used is from Stanford University Medical Center dataset. You can download it from https://stanfordaimi.azurewebsites.net/datasets/3a7548a4-8f65-4ab7-85fa-3d68c9efc1bd.

And the tabular data is already provided in this link.

@InProceedings{guo2024pemvcnet,
    author    = {Guo, Zhaoxin and Wang, Zhipeng and Ge, Ruiquan and Yu, Jianxun and Qin, Feiwei and Tian, Yuan and Peng, Yuqing and Li, Yonghong and Wang, Changmiao},
    title     = {PE-MVCNet: Multi-view and Cross-modal Fusion Network for Pulmonary Embolism Prediction},
    booktitle = {International Symposium on Biomedical Imaging -- ISBI 2024},
    month     = {May},
    year      = {2024},
    url       = {}
}