robertwyq / panoocc Goto Github PK

I see that you compared the detection accuracy in the ablation experiment of the paper, but the code did not test the detection accuracy during the testing process, may I ask if there is any open source code?

Some of the previous methods didn't use camera_mask during training, but used it during eval, right?

I would like to ask in which file the Refine Module is defined?

I want to know if there is a problem here? Shouldn't it be from below

 for i in range(bs):
                lidar_to_ego = kwargs[i]['lidar2ego_transformation']
                curr_ego_to_global = kwargs[i]['ego2global_transform_lst'][-1]
                curr_lidar_to_global = curr_ego_to_global @ lidar_to_ego
                curr_grid_in_prev_frame_lst = []
                for j in range(len_queue):
                    prev_ego_to_global = kwargs[i]['ego2global_transform_lst'][j]
                    prev_lidar_to_global = prev_ego_to_global @ lidar_to_ego

                    prev_lidar_to_curr_lidar =  np.linalg.inv(curr_lidar_to_global) @ prev_lidar_to_global 
                    curr_lidar_to_prev_lidar = np.linalg.inv(prev_lidar_to_curr_lidar)

How to visualize the Occupancy results in .npz format？

Thanks for your excellent work, will you release the chekpoints, and when will you release them. Very thanks.

Thank you for sharing your excellent work. Do you have any data you can share on inference performance time?

May I ask what changes I need to make to the code when testing with a single GPU？

Hi, thanks for your fantastic work! I have a question about the implementation of the voxel self-attention.

The paper writes ``These sampling points share the same height $z_k$, but with different learnable offsets for $(x_i^m , y_j^m )$. This encourages the voxel queries to interact in the BEV plane". To my understanding, this operation equals to split the voxel features into bev slices through the height dimension and perform deformable attention seperately for each bev slices, is this right? I wonder why not using 3D deformable attention directly?

By the way, I think the implementation of voxel self-attention may also has some problems.

Is there something wrong with this implementation? Or maybe there are some details that I didn’t notice.

Hi
In docker container, After completing the environment setting, I meet this error

[W ProcessGroupNCCL.cpp:1569] Rank 0 using best-guess GPU 0 to perform barrier as devices used by this process are currently unknown. This can potentially cause a hang if this rank to GPU mapping is incorrect.Specify device_ids in barrier() to force use of a particular device.
Traceback (most recent call last):
Traceback (most recent call last):
File "./tools/train.py", line 256, in
File "./tools/train.py", line 256, in
main()
File "./tools/train.py", line 216, in main
main()
File "./tools/train.py", line 216, in main
model.init_weights()
File "/root/anaconda3/envs/occ/lib/python3.8/site-packages/mmcv/runner/base_module.py", line 117, in init_weights
model.init_weights()
File "/root/anaconda3/envs/occ/lib/python3.8/site-packages/mmcv/runner/base_module.py", line 117, in init_weights
m.init_weights()
File "/root/anaconda3/envs/occ/lib/python3.8/site-packages/mmcv/runner/base_module.py", line 106, in init_weights
m.init_weights()
File "/root/anaconda3/envs/occ/lib/python3.8/site-packages/mmcv/runner/base_module.py", line 106, in init_weights
initialize(self, self.init_cfg)
File "/root/anaconda3/envs/occ/lib/python3.8/site-packages/mmcv/cnn/utils/weight_init.py", line 612, in initialize
initialize(self, self.init_cfg)
File "/root/anaconda3/envs/occ/lib/python3.8/site-packages/mmcv/cnn/utils/weight_init.py", line 612, in initialize
_initialize(module, cp_cfg)
File "/root/anaconda3/envs/occ/lib/python3.8/site-packages/mmcv/cnn/utils/weight_init.py", line 517, in _initialize
_initialize(module, cp_cfg)
File "/root/anaconda3/envs/occ/lib/python3.8/site-packages/mmcv/cnn/utils/weight_init.py", line 517, in _initialize
func(module)
File "/root/anaconda3/envs/occ/lib/python3.8/site-packages/mmcv/cnn/utils/weight_init.py", line 489, in call
func(module)
File "/root/anaconda3/envs/occ/lib/python3.8/site-packages/mmcv/cnn/utils/weight_init.py", line 489, in call
load_checkpoint(
File "/root/anaconda3/envs/occ/lib/python3.8/site-packages/mmcv/runner/checkpoint.py", line 531, in load_checkpoint
load_checkpoint(
File "/root/anaconda3/envs/occ/lib/python3.8/site-packages/mmcv/runner/checkpoint.py", line 531, in load_checkpoint
checkpoint = _load_checkpoint(filename, map_location, logger)
File "/root/anaconda3/envs/occ/lib/python3.8/site-packages/mmcv/runner/checkpoint.py", line 470, in _load_checkpoint
checkpoint = _load_checkpoint(filename, map_location, logger)
File "/root/anaconda3/envs/occ/lib/python3.8/site-packages/mmcv/runner/checkpoint.py", line 470, in _load_checkpoint
return CheckpointLoader.load_checkpoint(filename, map_location, logger)
File "/root/anaconda3/envs/occ/lib/python3.8/site-packages/mmcv/runner/checkpoint.py", line 249, in load_checkpoint
return CheckpointLoader.load_checkpoint(filename, map_location, logger)
File "/root/anaconda3/envs/occ/lib/python3.8/site-packages/mmcv/runner/checkpoint.py", line 249, in load_checkpoint
return checkpoint_loader(filename, map_location)
File "/root/anaconda3/envs/occ/lib/python3.8/site-packages/mmcv/runner/checkpoint.py", line 390, in load_from_torchvision
return checkpoint_loader(filename, map_location)
File "/root/anaconda3/envs/occ/lib/python3.8/site-packages/mmcv/runner/checkpoint.py", line 390, in load_from_torchvision
return load_from_http(model_urls[model_name], map_location=map_location)
File "/root/anaconda3/envs/occ/lib/python3.8/site-packages/mmcv/runner/checkpoint.py", line 290, in load_from_http
return load_from_http(model_urls[model_name], map_location=map_location)
File "/root/anaconda3/envs/occ/lib/python3.8/site-packages/mmcv/runner/checkpoint.py", line 290, in load_from_http
torch.distributed.barrier()
File "/root/anaconda3/envs/occ/lib/python3.8/site-packages/torch/distributed/distributed_c10d.py", line 2538, in barrier
torch.distributed.barrier()
File "/root/anaconda3/envs/occ/lib/python3.8/site-packages/torch/distributed/distributed_c10d.py", line 2538, in barrier
work = default_pg.barrier(opts=opts)
RuntimeError: NCCL error in: ../torch/lib/c10d/ProcessGroupNCCL.cpp:911, invalid usage, NCCL version 2.7.8
ncclInvalidUsage: This usually reflects invalid usage of NCCL library (such as too many async ops, too many collectives at once, mixing streams in a group, etc).
work = default_pg.barrier(opts=opts)
RuntimeError: NCCL error in: ../torch/lib/c10d/ProcessGroupNCCL.cpp:911, invalid usage, NCCL version 2.7.8
ncclInvalidUsage: This usually reflects invalid usage of NCCL library (such as too many async ops, too many collectives at once, mixing streams in a group, etc).
ERROR:torch.distributed.elastic.multiprocessing.api:failed (exitcode: 1) local_rank: 0 (pid: 23199) of binary: /root/anaconda3/envs/occ/bin/python
Traceback (most recent call last):
File "/root/anaconda3/envs/occ/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/root/anaconda3/envs/occ/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/root/anaconda3/envs/occ/lib/python3.8/site-packages/torch/distributed/launch.py", line 193, in
main()
File "/root/anaconda3/envs/occ/lib/python3.8/site-packages/torch/distributed/launch.py", line 189, in main
launch(args)
File "/root/anaconda3/envs/occ/lib/python3.8/site-packages/torch/distributed/launch.py", line 174, in launch
run(args)
File "/root/anaconda3/envs/occ/lib/python3.8/site-packages/torch/distributed/run.py", line 689, in run
elastic_launch(
File "/root/anaconda3/envs/occ/lib/python3.8/site-packages/torch/distributed/launcher/api.py", line 116, in call
return launch_agent(self._config, self._entrypoint, list(args))
File "/root/anaconda3/envs/occ/lib/python3.8/site-packages/torch/distributed/launcher/api.py", line 244, in launch_agent
raise ChildFailedError(
torch.distributed.elastic.multiprocessing.errors.ChildFailedError:

I think this error is related with NCCL, but I can't find solution
I use single GPU RTX 3080, and I just follow your install.md. So I, think all setting is same with your environment.

Please Reply me plz.
Thank you.