cannot run evaluation for semantic segmentation with the checkpoint about tokenfusion HOT 4 CLOSED

change python3 main.py --gpu 0 --resume ./pretrained_models/model-best.pth.tar --evaluate to python3 main.py --gpu 0 --backbone mit_b3 --resume ./pretrained_models/model-best.pth.tar --evaluate solve the evaluation problem, because the default backbone in main.py is mit_b0

from tokenfusion.

After commenting out the part of the PyTorch source code that checks for unexpected keys when loading the checkpoint, I was able to proceed with inference normally. However, the inference results were very poor. The pre-trained model used was provided in the README.

img = make_validation_img(inputs[0].data.cpu().numpy(),  # color image
                          inputs[1].data.cpu().numpy(),   # depth image
                          sample['mask'].data.cpu().numpy(),  # ground truth mask
                          output[np.newaxis,:])   # predicted semantic mask

from tokenfusion.

change the code to v.load_state_dict(ckpt[k], strict=False) can solve the "unexpected keys" problems.

But the evaluation result is quite bad:

Epoch 0    (rgb)     glob_acc=11.14    mean_acc=5.10     IoU=1.81             (best)
Epoch 0    (depth)   glob_acc=9.72     mean_acc=3.85     IoU=1.39         
Epoch 0    (ens)     glob_acc=14.35    mean_acc=4.81     IoU=1.86             (best)

from tokenfusion.

if you load the chechpoint properly, you can get the right result.

from tokenfusion.