vincrichard / efficientps Goto Github PK

Hey @vincrichard,
I'm really close at finishing my project but one thing I still would like to ask you is why did you use train id instead of eval id in your code? I think that it doesn't matter which id you use internally as long as they correspond to the same semantic class, but I am not sure since I'm still a novice in ML. Does it just make indexing easier or the label indices have to be in ascending order for the model to work properly?

Thanks a lot,
Dang

Hey @vincrichard,
thanks a lot for helping me the last time. I am trying to make use of your code in my own project but I stumbled on something that is really confusing me. Why did you multiply the id of thing instances by 1000 to create the canvas?
canvas[mask] = instance_train_id_to_eval_id[cls] * 1000 + nb_instance

Regards
Dang

在训练的时候出现'loss_rpn_loc': tensor(nan, device='cuda:0', grad_fn=) ，没找到哪里的问题

HI, @vincrichard thank you for this amazing work. Is there any way I can load a pre-trained model from your implementation of EfficientPS before training? Thanks a lot.

Hi, I tried running the official code of EfficientPS(from their github) on google colab but because of restricted cuda, pytorch versions and versions of other dependencies too, I kept facing errors.
Then I came across your implementation. And while setting up did not found any error till now. But I haven't yet tried training. I want to train another dataset (not cityscapes) . If you could please help me how should I change this code to train on a custom dataset - mainly the panoptic dataloader part.
Thanks!

Hi I have looked everywhere and done everything that made sense to me to get rid of this error and nothing seems to work. It would be a huge help if you could help me out with this.

I am working with:
pytorch = 1.13.1
Cuda = 11.7
python = 3.9.13

The error:

Traceback (most recent call last):
File "C:\Users\singh\Downloads\EfficientPS\impl\EfficientPS\train_net.py", line 178, in
main()
File "C:\Users\singh\Downloads\EfficientPS\impl\EfficientPS\train_net.py", line 175, in main
trainer.fit(efficientps, train_loader, val_dataloaders=valid_loader)
File "C:\Users\singh\anaconda3\envs\efficientPS_env\lib\site-packages\pytorch_lightning\trainer\trainer.py", line 521, in fit
call._call_and_handle_interrupt(
File "C:\Users\singh\anaconda3\envs\efficientPS_env\lib\site-packages\pytorch_lightning\trainer\call.py", line 44, in _call_and_handle_interrupt
return trainer_fn(*args, **kwargs)
File "C:\Users\singh\anaconda3\envs\efficientPS_env\lib\site-packages\pytorch_lightning\trainer\trainer.py", line 560, in _fit_impl
self._run(model, ckpt_path=ckpt_path)
File "C:\Users\singh\anaconda3\envs\efficientPS_env\lib\site-packages\pytorch_lightning\trainer\trainer.py", line 936, in _run
results = self._run_stage()
File "C:\Users\singh\anaconda3\envs\efficientPS_env\lib\site-packages\pytorch_lightning\trainer\trainer.py", line 979, in _run_stage
self.fit_loop.run()
File "C:\Users\singh\anaconda3\envs\efficientPS_env\lib\site-packages\pytorch_lightning\loops\fit_loop.py", line 201, in run
self.advance()
File "C:\Users\singh\anaconda3\envs\efficientPS_env\lib\site-packages\pytorch_lightning\loops\fit_loop.py", line 354, in advance
self.epoch_loop.run(self._data_fetcher)
File "C:\Users\singh\anaconda3\envs\efficientPS_env\lib\site-packages\pytorch_lightning\loops\training_epoch_loop.py", line 133, in run
self.advance(data_fetcher)
File "C:\Users\singh\anaconda3\envs\efficientPS_env\lib\site-packages\pytorch_lightning\loops\training_epoch_loop.py", line 218, in advance
batch_output = self.automatic_optimization.run(trainer.optimizers[0], kwargs)
File "C:\Users\singh\anaconda3\envs\efficientPS_env\lib\site-packages\pytorch_lightning\loops\optimization\automatic.py", line 185, in run
self._optimizer_step(kwargs.get("batch_idx", 0), closure)
File "C:\Users\singh\anaconda3\envs\efficientPS_env\lib\site-packages\pytorch_lightning\loops\optimization\automatic.py", line 261, in _optimizer_step
call._call_lightning_module_hook(
File "C:\Users\singh\anaconda3\envs\efficientPS_env\lib\site-packages\pytorch_lightning\trainer\call.py", line 143, in _call_lightning_module_hook
output = fn(*args, **kwargs)
TypeError: optimizer_step() missing 1 required positional argument: 'closure'
Epoch 0: 0%| | 0/992 [00:19<?, ?it/s]

Hi @vincrichard,
thank you for your word. I currently have an issue that I can't solve. I would appreciate it if you could help me with it.

Traceback (most recent call last):
File "/home/dangnguyen/myProjects/EfficientPS_vinc/train_net.py", line 164, in
main()
File "/home/dangnguyen/myProjects/EfficientPS_vinc/train_net.py", line 161, in main
trainer.fit(efficientps, train_loader, val_dataloaders=valid_loader)
File "/home/dangnguyen/myProjects/EfficientPS_vinc/venv/lib/python3.9/site-packages/pytorch_lightning/trainer/trainer.py", line 740, in fit
self._call_and_handle_interrupt(
File "/home/dangnguyen/myProjects/EfficientPS_vinc/venv/lib/python3.9/site-packages/pytorch_lightning/trainer/trainer.py", line 685, in _call_and_handle_interrupt
return trainer_fn(*args, **kwargs)
File "/home/dangnguyen/myProjects/EfficientPS_vinc/venv/lib/python3.9/site-packages/pytorch_lightning/trainer/trainer.py", line 777, in _fit_impl
self._run(model, ckpt_path=ckpt_path)
File "/home/dangnguyen/myProjects/EfficientPS_vinc/venv/lib/python3.9/site-packages/pytorch_lightning/trainer/trainer.py", line 1199, in _run
self._dispatch()
File "/home/dangnguyen/myProjects/EfficientPS_vinc/venv/lib/python3.9/site-packages/pytorch_lightning/trainer/trainer.py", line 1279, in _dispatch
self.training_type_plugin.start_training(self)
File "/home/dangnguyen/myProjects/EfficientPS_vinc/venv/lib/python3.9/site-packages/pytorch_lightning/plugins/training_type/training_type_plugin.py", line 202, in start_training
self._results = trainer.run_stage()
File "/home/dangnguyen/myProjects/EfficientPS_vinc/venv/lib/python3.9/site-packages/pytorch_lightning/trainer/trainer.py", line 1289, in run_stage
return self._run_train()
File "/home/dangnguyen/myProjects/EfficientPS_vinc/venv/lib/python3.9/site-packages/pytorch_lightning/trainer/trainer.py", line 1319, in _run_train
self.fit_loop.run()
File "/home/dangnguyen/myProjects/EfficientPS_vinc/venv/lib/python3.9/site-packages/pytorch_lightning/loops/base.py", line 145, in run
self.advance(*args, **kwargs)
File "/home/dangnguyen/myProjects/EfficientPS_vinc/venv/lib/python3.9/site-packages/pytorch_lightning/loops/fit_loop.py", line 234, in advance
self.epoch_loop.run(data_fetcher)
File "/home/dangnguyen/myProjects/EfficientPS_vinc/venv/lib/python3.9/site-packages/pytorch_lightning/loops/base.py", line 145, in run
self.advance(*args, **kwargs)
File "/home/dangnguyen/myProjects/EfficientPS_vinc/venv/lib/python3.9/site-packages/pytorch_lightning/loops/epoch/training_epoch_loop.py", line 193, in advance
batch_output = self.batch_loop.run(batch, batch_idx)
File "/home/dangnguyen/myProjects/EfficientPS_vinc/venv/lib/python3.9/site-packages/pytorch_lightning/loops/base.py", line 145, in run
self.advance(*args, **kwargs)
File "/home/dangnguyen/myProjects/EfficientPS_vinc/venv/lib/python3.9/site-packages/pytorch_lightning/loops/batch/training_batch_loop.py", line 88, in advance
outputs = self.optimizer_loop.run(split_batch, optimizers, batch_idx)
File "/home/dangnguyen/myProjects/EfficientPS_vinc/venv/lib/python3.9/site-packages/pytorch_lightning/loops/base.py", line 145, in run
self.advance(*args, **kwargs)
File "/home/dangnguyen/myProjects/EfficientPS_vinc/venv/lib/python3.9/site-packages/pytorch_lightning/loops/optimization/optimizer_loop.py", line 215, in advance
result = self._run_optimization(
File "/home/dangnguyen/myProjects/EfficientPS_vinc/venv/lib/python3.9/site-packages/pytorch_lightning/loops/optimization/optimizer_loop.py", line 266, in _run_optimization
self._optimizer_step(optimizer, opt_idx, batch_idx, closure)
File "/home/dangnguyen/myProjects/EfficientPS_vinc/venv/lib/python3.9/site-packages/pytorch_lightning/loops/optimization/optimizer_loop.py", line 378, in _optimizer_step
lightning_module.optimizer_step(
File "/home/dangnguyen/myProjects/EfficientPS_vinc/efficientps/model.py", line 131, in optimizer_step
optimizer.step(closure=closure)
File "/home/dangnguyen/myProjects/EfficientPS_vinc/venv/lib/python3.9/site-packages/pytorch_lightning/core/optimizer.py", line 164, in step
trainer.accelerator.optimizer_step(self._optimizer, self._optimizer_idx, closure, **kwargs)
File "/home/dangnguyen/myProjects/EfficientPS_vinc/venv/lib/python3.9/site-packages/pytorch_lightning/accelerators/accelerator.py", line 336, in optimizer_step
self.precision_plugin.optimizer_step(model, optimizer, opt_idx, closure, **kwargs)
File "/home/dangnguyen/myProjects/EfficientPS_vinc/venv/lib/python3.9/site-packages/pytorch_lightning/plugins/precision/native_amp.py", line 85, in optimizer_step
closure_result = closure()
File "/home/dangnguyen/myProjects/EfficientPS_vinc/venv/lib/python3.9/site-packages/pytorch_lightning/loops/optimization/optimizer_loop.py", line 160, in call
self._result = self.closure(*args, **kwargs)
File "/home/dangnguyen/myProjects/EfficientPS_vinc/venv/lib/python3.9/site-packages/pytorch_lightning/loops/optimization/optimizer_loop.py", line 142, in closure
step_output = self._step_fn()
File "/home/dangnguyen/myProjects/EfficientPS_vinc/venv/lib/python3.9/site-packages/pytorch_lightning/loops/optimization/optimizer_loop.py", line 435, in _training_step
training_step_output = self.trainer.accelerator.training_step(step_kwargs)
File "/home/dangnguyen/myProjects/EfficientPS_vinc/venv/lib/python3.9/site-packages/pytorch_lightning/accelerators/accelerator.py", line 216, in training_step
return self.training_type_plugin.training_step(*step_kwargs.values())
File "/home/dangnguyen/myProjects/EfficientPS_vinc/venv/lib/python3.9/site-packages/pytorch_lightning/plugins/training_type/training_type_plugin.py", line 213, in training_step
return self.model.training_step(*args, **kwargs)
File "/home/dangnguyen/myProjects/EfficientPS_vinc/efficientps/model.py", line 41, in training_step
_, loss = self.shared_step(batch)
File "/home/dangnguyen/myProjects/EfficientPS_vinc/efficientps/model.py", line 52, in shared_step
pyramid_features = self.fpn(features)
File "/home/dangnguyen/myProjects/EfficientPS_vinc/venv/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
return forward_call(*input, **kwargs)
File "/home/dangnguyen/myProjects/EfficientPS_vinc/efficientps/fpn/two_way_fpn.py", line 155, in forward
b_up_x32 = self.conv_b_up_x32(b_up_x32)
File "/home/dangnguyen/myProjects/EfficientPS_vinc/venv/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
return forward_call(*input, **kwargs)
File "/home/dangnguyen/myProjects/EfficientPS_vinc/venv/lib/python3.9/site-packages/torch/nn/modules/conv.py", line 446, in forward
return self._conv_forward(input, self.weight, self.bias)
File "/home/dangnguyen/myProjects/EfficientPS_vinc/venv/lib/python3.9/site-packages/torch/nn/modules/conv.py", line 442, in _conv_forward
return F.conv2d(input, weight, bias, self.stride,
RuntimeError: Given groups=1, weight of size [256, 2048, 1, 1], expected input[3, 512, 16, 32] to have 2048 channels, but got 512 channels instead
Epoch 0: 0%| | 0/1159 [00:05<?, ?it/s]