jhultman / vision3d Goto Github PK

Research platform for 3D object detection in PyTorch.

License: MIT License

Python 78.66% C++ 13.82% Cuda 7.52%

3d-deep-learning object-detection lidar point-cloud pytorch voxel detection codebase perception autonomous-vehicles real-time

vision3d's Introduction

Vision 3D

A clean, easy-to-use PyTorch library for lidar perception.

Project goals

Emphasis on clean code (no 1,000 LOC functions).
General 3D detection library (easy to extend to new models and datasets).

Status

This project is not under active development.
Implementation of SECOND is complete.
Implementation of PV-RCNN is partially completed.
These forks (one, two) have shown some promise in training on other datasets (NuScenes, and proprietary lidar data).

Usage

See inference.py and train.py. To train, need to first start a visdom server using command visdom to enable train loss monitoring. (Requires visdom python package to be installed).

Installation

See install.md.

Sample results on validation data (KITTI)

Citing

If you find this work helpful in your research, please consider starring this repo and citing:

@article{hultman2020vision3d,
  author={Jacob Hultman},
  title={vision3d},
  journal={https://github.com/jhultman/vision3d},
  year={2020}
}

Contributions

Contributions are welcome. Please post an issue if you find any bugs.

Acknowledgements and licensing

Please see license.md. Note that the code in vision3d/ops is largely from detectron2 and hence is subject to the Apache license.

vision3d's People

Contributors

Stargazers

Watchers

Forkers

yamin05114 collector-m turboxin you4ever robin2002 nnu-gisa chaomath allenpeng0209 eraofelix wyjforwjy muzi2045 ansonyanxin fulin2019 chaos1992 bruinxiong liuguoyou zhengkai930504 chasingw bilybill turingdriving electronicyh 787674247 yuxguo adodd202 maurock mengdejiang avatazjoe smallmunich dd-iuonac rj2019 thuyngch dinglinmin ngoc-nguyen-nis liyang0522 viraj96 yi-shi94 anhvth luke-dinh dayoungmm balaharivignesh arqam-ai dj-boy aiedward enginbozkurt zhen12079

vision3d's Issues

pre-trained network?

Can you please upload a pre-trained network or at least show us how to train? Thanks!

Error about running "train.py"

I tried to run 'train.py' to train with KITTI dataset but encountered the following issue:

Setting up a new session...
Loading cached annotations: ../data/cache/train.pkl
Found cached database: ../data/cache/database.pkl
Epoch 0:   0%|                                          | 0/928 [00:00<?, ?it/s]Traceback (most recent call last):
  File "/root/csl_project/PV-RCNN/pvrcnn/train.py", line 105, in <module>
    main()
  File "/root/csl_project/PV-RCNN/pvrcnn/train.py", line 94, in main
    scheduler, loss_fn, cfg.TRAIN.EPOCHS, start_epoch)
  File "/root/csl_project/PV-RCNN/pvrcnn/train.py", line 63, in train_model
    out = model(item)
  File "/usr/local/miniconda3/envs/pvrcnn/lib/python3.7/site-packages/torch/nn/modules/module.py", line 532, in __call__
    result = self.forward(*input, **kwargs)
  File "/root/csl_project/PV-RCNN/pvrcnn/detector/second.py", line 27, in forward
    features = self.feature_extract(item)
  File "/root/csl_project/PV-RCNN/pvrcnn/detector/second.py", line 22, in feature_extract
    features = self.cnn(features, item['coordinates'], item['batch_size'])
  File "/usr/local/miniconda3/envs/pvrcnn/lib/python3.7/site-packages/torch/nn/modules/module.py", line 532, in __call__
    result = self.forward(*input, **kwargs)
  File "/root/csl_project/PV-RCNN/pvrcnn/detector/second.py", line 45, in forward
    x = self.blocks(x)
  File "/usr/local/miniconda3/envs/pvrcnn/lib/python3.7/site-packages/torch/nn/modules/module.py", line 532, in __call__
    result = self.forward(*input, **kwargs)
  File "/usr/local/miniconda3/envs/pvrcnn/lib/python3.7/site-packages/spconv/modules.py", line 123, in forward
    input = module(input)
  File "/usr/local/miniconda3/envs/pvrcnn/lib/python3.7/site-packages/torch/nn/modules/module.py", line 532, in __call__
    result = self.forward(*input, **kwargs)
  File "/usr/local/miniconda3/envs/pvrcnn/lib/python3.7/site-packages/spconv/modules.py", line 123, in forward
    input = module(input)
  File "/usr/local/miniconda3/envs/pvrcnn/lib/python3.7/site-packages/torch/nn/modules/module.py", line 532, in __call__
    result = self.forward(*input, **kwargs)
  File "/usr/local/miniconda3/envs/pvrcnn/lib/python3.7/site-packages/spconv/modules.py", line 123, in forward
    input = module(input)
  File "/usr/local/miniconda3/envs/pvrcnn/lib/python3.7/site-packages/torch/nn/modules/module.py", line 532, in __call__
    result = self.forward(*input, **kwargs)
  File "/usr/local/miniconda3/envs/pvrcnn/lib/python3.7/site-packages/spconv/conv.py", line 161, in forward
    outids.shape[0])
  File "/usr/local/miniconda3/envs/pvrcnn/lib/python3.7/site-packages/spconv/functional.py", line 83, in forward
    return ops.indice_conv(features, filters, indice_pairs, indice_pair_num, num_activate_out, False, True)
  File "/usr/local/miniconda3/envs/pvrcnn/lib/python3.7/site-packages/spconv/ops.py", line 112, in indice_conv
    int(inverse), int(subm))
RuntimeError: CUDA error: CUBLAS_STATUS_NOT_INITIALIZED when calling `cublasCreate(handle)` (createCublasHandle at /opt/conda/conda-bld/pytorch_1579022060824/work/aten/src/ATen/cuda/CublasHandlePool.cpp:8)
frame #0: c10::Error::Error(c10::SourceLocation, std::string const&) + 0x47 (0x7ff3eafa5627 in /usr/local/miniconda3/envs/pvrcnn/lib/python3.7/site-packages/torch/lib/libc10.so)
frame #1: <unknown function> + 0x4173335 (0x7ff3f0e43335 in /usr/local/miniconda3/envs/pvrcnn/lib/python3.7/site-packages/torch/lib/libtorch.so)
frame #2: at::cuda::getCurrentCUDABlasHandle() + 0x458 (0x7ff3f0e43c18 in /usr/local/miniconda3/envs/pvrcnn/lib/python3.7/site-packages/torch/lib/libtorch.so)
frame #3: <unknown function> + 0x416b092 (0x7ff3f0e3b092 in /usr/local/miniconda3/envs/pvrcnn/lib/python3.7/site-packages/torch/lib/libtorch.so)
frame #4: THCudaBlas_Sgemm + 0x7e (0x7ff3f1248a3e in /usr/local/miniconda3/envs/pvrcnn/lib/python3.7/site-packages/torch/lib/libtorch.so)
frame #5: <unknown function> + 0x45c9cfb (0x7ff3f1299cfb in /usr/local/miniconda3/envs/pvrcnn/lib/python3.7/site-packages/torch/lib/libtorch.so)
frame #6: THCudaTensor_addmm + 0x57 (0x7ff3f129e607 in /usr/local/miniconda3/envs/pvrcnn/lib/python3.7/site-packages/torch/lib/libtorch.so)
frame #7: <unknown function> + 0x4258914 (0x7ff3f0f28914 in /usr/local/miniconda3/envs/pvrcnn/lib/python3.7/site-packages/torch/lib/libtorch.so)
frame #8: <unknown function> + 0x41c5fa2 (0x7ff3f0e95fa2 in /usr/local/miniconda3/envs/pvrcnn/lib/python3.7/site-packages/torch/lib/libtorch.so)
frame #9: <unknown function> + 0x159f45d (0x7ff3ee26f45d in /usr/local/miniconda3/envs/pvrcnn/lib/python3.7/site-packages/torch/lib/libtorch.so)
frame #10: <unknown function> + 0x38e7265 (0x7ff3f05b7265 in /usr/local/miniconda3/envs/pvrcnn/lib/python3.7/site-packages/torch/lib/libtorch.so)
frame #11: <unknown function> + 0x1f543 (0x7ff3cfd2d543 in /usr/local/miniconda3/envs/pvrcnn/lib/python3.7/site-packages/spconv/libspconv.so)
frame #12: at::Tensor spconv::indiceConv<float>(at::Tensor, at::Tensor, at::Tensor, at::Tensor, long, long, long) + 0x9f3 (0x7ff3cfd460f3 in /usr/local/miniconda3/envs/pvrcnn/lib/python3.7/site-packages/spconv/libspconv.so)
frame #13: c10::guts::infer_function_traits_t::return_type c10::detail::call_functor_with_args_from_stack_<c10::detail::WrapRuntimeKernelFunctor_<at::Tensor (*)(at::Tensor, at::Tensor, at::Tensor, at::Tensor, long, long, long), at::Tensor, c10::guts::typelist::typelist<at::Tensor, at::Tensor, at::Tensor, at::Tensor, long, long, long> >, true, 0ul, 1ul, 2ul, 3ul, 4ul, 5ul, 6ul>(c10::detail::WrapRuntimeKernelFunctor_<at::Tensor (*)(at::Tensor, at::Tensor, at::Tensor, at::Tensor, long, long, long), at::Tensor, c10::guts::typelist::typelist<at::Tensor, at::Tensor, at::Tensor, at::Tensor, long, long, long> >*, std::vector<c10::IValue, std::allocator<c10::IValue> >*, std::integer_sequence<unsigned long, 0ul, 1ul, 2ul, 3ul, 4ul, 5ul, 6ul>) + 0x161 (0x7ff3cfd4d791 in /usr/local/miniconda3/envs/pvrcnn/lib/python3.7/site-packages/spconv/libspconv.so)
frame #14: c10::detail::wrap_kernel_functor_boxed<c10::detail::WrapRuntimeKernelFunctor_<at::Tensor (*)(at::Tensor, at::Tensor, at::Tensor, at::Tensor, long, long, long), at::Tensor, c10::guts::typelist::typelist<at::Tensor, at::Tensor, at::Tensor, at::Tensor, long, long, long> >, true, void>::call(c10::OperatorKernel*, std::vector<c10::IValue, std::allocator<c10::IValue> >*) + 0x44 (0x7ff3cfd56864 in /usr/local/miniconda3/envs/pvrcnn/lib/python3.7/site-packages/spconv/libspconv.so)
frame #15: <unknown function> + 0x2fa9fc9 (0x7ff3efc79fc9 in /usr/local/miniconda3/envs/pvrcnn/lib/python3.7/site-packages/torch/lib/libtorch.so)
frame #16: <unknown function> + 0x3ca7507 (0x7ff3f0977507 in /usr/local/miniconda3/envs/pvrcnn/lib/python3.7/site-packages/torch/lib/libtorch.so)
frame #17: <unknown function> + 0x6ea9a7 (0x7ff41d0289a7 in /usr/local/miniconda3/envs/pvrcnn/lib/python3.7/site-packages/torch/lib/libtorch_python.so)
frame #18: <unknown function> + 0x6b4fe4 (0x7ff41cff2fe4 in /usr/local/miniconda3/envs/pvrcnn/lib/python3.7/site-packages/torch/lib/libtorch_python.so)
frame #19: <unknown function> + 0x28ba06 (0x7ff41cbc9a06 in /usr/local/miniconda3/envs/pvrcnn/lib/python3.7/site-packages/torch/lib/libtorch_python.so)
frame #20: _PyMethodDef_RawFastCallKeywords + 0x264 (0x559f7471fab4 in /usr/local/miniconda3/envs/pvrcnn/bin/python3.7m)
frame #21: _PyCFunction_FastCallKeywords + 0x21 (0x559f7471fbd1 in /usr/local/miniconda3/envs/pvrcnn/bin/python3.7m)
frame #22: _PyEval_EvalFrameDefault + 0x4ecb (0x559f7478657b in /usr/local/miniconda3/envs/pvrcnn/bin/python3.7m)
frame #23: _PyEval_EvalCodeWithName + 0x2f9 (0x559f746cb389 in /usr/local/miniconda3/envs/pvrcnn/bin/python3.7m)
frame #24: _PyFunction_FastCallKeywords + 0x325 (0x559f7471f255 in /usr/local/miniconda3/envs/pvrcnn/bin/python3.7m)
frame #25: _PyEval_EvalFrameDefault + 0x4b39 (0x559f747861e9 in /usr/local/miniconda3/envs/pvrcnn/bin/python3.7m)
frame #26: _PyFunction_FastCallDict + 0x10b (0x559f746cc3fb in /usr/local/miniconda3/envs/pvrcnn/bin/python3.7m)
frame #27: THPFunction_apply(_object*, _object*) + 0xa0f (0x7ff41cfb4a3f in /usr/local/miniconda3/envs/pvrcnn/lib/python3.7/site-packages/torch/lib/libtorch_python.so)
frame #28: _PyMethodDef_RawFastCallKeywords + 0x1e0 (0x559f7471fa30 in /usr/local/miniconda3/envs/pvrcnn/bin/python3.7m)
frame #29: _PyCFunction_FastCallKeywords + 0x21 (0x559f7471fbd1 in /usr/local/miniconda3/envs/pvrcnn/bin/python3.7m)
frame #30: _PyEval_EvalFrameDefault + 0x4ecb (0x559f7478657b in /usr/local/miniconda3/envs/pvrcnn/bin/python3.7m)
frame #31: _PyFunction_FastCallDict + 0x10b (0x559f746cc3fb in /usr/local/miniconda3/envs/pvrcnn/bin/python3.7m)
frame #32: _PyObject_Call_Prepend + 0x63 (0x559f746eba73 in /usr/local/miniconda3/envs/pvrcnn/bin/python3.7m)
frame #33: PyObject_Call + 0x6e (0x559f746ddfde in /usr/local/miniconda3/envs/pvrcnn/bin/python3.7m)
frame #34: _PyEval_EvalFrameDefault + 0x1e9d (0x559f7478354d in /usr/local/miniconda3/envs/pvrcnn/bin/python3.7m)
frame #35: _PyEval_EvalCodeWithName + 0x2f9 (0x559f746cb389 in /usr/local/miniconda3/envs/pvrcnn/bin/python3.7m)
frame #36: _PyFunction_FastCallDict + 0x1d5 (0x559f746cc4c5 in /usr/local/miniconda3/envs/pvrcnn/bin/python3.7m)
frame #37: _PyObject_Call_Prepend + 0x63 (0x559f746eba73 in /usr/local/miniconda3/envs/pvrcnn/bin/python3.7m)
frame #38: <unknown function> + 0x17d27a (0x559f7473327a in /usr/local/miniconda3/envs/pvrcnn/bin/python3.7m)
frame #39: _PyObject_FastCallKeywords + 0x48b (0x559f747342db in /usr/local/miniconda3/envs/pvrcnn/bin/python3.7m)
frame #40: _PyEval_EvalFrameDefault + 0x4a96 (0x559f74786146 in /usr/local/miniconda3/envs/pvrcnn/bin/python3.7m)
frame #41: _PyFunction_FastCallDict + 0x10b (0x559f746cc3fb in /usr/local/miniconda3/envs/pvrcnn/bin/python3.7m)
frame #42: _PyObject_Call_Prepend + 0x63 (0x559f746eba73 in /usr/local/miniconda3/envs/pvrcnn/bin/python3.7m)
frame #43: PyObject_Call + 0x6e (0x559f746ddfde in /usr/local/miniconda3/envs/pvrcnn/bin/python3.7m)
frame #44: _PyEval_EvalFrameDefault + 0x1e9d (0x559f7478354d in /usr/local/miniconda3/envs/pvrcnn/bin/python3.7m)
frame #45: _PyEval_EvalCodeWithName + 0x2f9 (0x559f746cb389 in /usr/local/miniconda3/envs/pvrcnn/bin/python3.7m)
frame #46: _PyFunction_FastCallDict + 0x1d5 (0x559f746cc4c5 in /usr/local/miniconda3/envs/pvrcnn/bin/python3.7m)
frame #47: _PyObject_Call_Prepend + 0x63 (0x559f746eba73 in /usr/local/miniconda3/envs/pvrcnn/bin/python3.7m)
frame #48: <unknown function> + 0x17d27a (0x559f7473327a in /usr/local/miniconda3/envs/pvrcnn/bin/python3.7m)
frame #49: _PyObject_FastCallKeywords + 0x48b (0x559f747342db in /usr/local/miniconda3/envs/pvrcnn/bin/python3.7m)
frame #50: _PyEval_EvalFrameDefault + 0x4a96 (0x559f74786146 in /usr/local/miniconda3/envs/pvrcnn/bin/python3.7m)
frame #51: _PyFunction_FastCallDict + 0x10b (0x559f746cc3fb in /usr/local/miniconda3/envs/pvrcnn/bin/python3.7m)
frame #52: _PyObject_Call_Prepend + 0x63 (0x559f746eba73 in /usr/local/miniconda3/envs/pvrcnn/bin/python3.7m)
frame #53: PyObject_Call + 0x6e (0x559f746ddfde in /usr/local/miniconda3/envs/pvrcnn/bin/python3.7m)
frame #54: _PyEval_EvalFrameDefault + 0x1e9d (0x559f7478354d in /usr/local/miniconda3/envs/pvrcnn/bin/python3.7m)
frame #55: _PyEval_EvalCodeWithName + 0x2f9 (0x559f746cb389 in /usr/local/miniconda3/envs/pvrcnn/bin/python3.7m)
frame #56: _PyFunction_FastCallDict + 0x1d5 (0x559f746cc4c5 in /usr/local/miniconda3/envs/pvrcnn/bin/python3.7m)
frame #57: _PyObject_Call_Prepend + 0x63 (0x559f746eba73 in /usr/local/miniconda3/envs/pvrcnn/bin/python3.7m)
frame #58: <unknown function> + 0x17d27a (0x559f7473327a in /usr/local/miniconda3/envs/pvrcnn/bin/python3.7m)
frame #59: _PyObject_FastCallKeywords + 0x48b (0x559f747342db in /usr/local/miniconda3/envs/pvrcnn/bin/python3.7m)
frame #60: _PyEval_EvalFrameDefault + 0x4a96 (0x559f74786146 in /usr/local/miniconda3/envs/pvrcnn/bin/python3.7m)
frame #61: _PyFunction_FastCallDict + 0x10b (0x559f746cc3fb in /usr/local/miniconda3/envs/pvrcnn/bin/python3.7m)
frame #62: _PyObject_Call_Prepend + 0x63 (0x559f746eba73 in /usr/local/miniconda3/envs/pvrcnn/bin/python3.7m)
frame #63: PyObject_Call + 0x6e (0x559f746ddfde in /usr/local/miniconda3/envs/pvrcnn/bin/python3.7m)

Epoch 0:   0%|                                          | 0/928 [01:07<?, ?it/s]

Process finished with exit code 1

I don't know how to solve this problem, could you help me?

Should grid point randomly sampled or evenly distributed

Hi @jhultman，thanks for sharing your implementation.

According to the original paper, the grid points are uniformly sampled within a proposal, I think "uniformly" here means evenly distributed in every dimension( from figure4 we can see the red grid points are not randomly sampled), and the name "grid" itself may also claim this point.

I think "uniformly" here is different from "uniform distribution" given by torch.rand

vision3d/pvrcnn/detector/roi_grid_pool.py

Line 59 in 0789508

(torch.rand((b, n, m, 3), device=boxes.device) - 0.5)

Hello author, I am a novice, can you write more detailed steps for training and testing this model, thank you

Could you please share the FPS benchmarking results of your experiments?

Abour the GPU memory using

Tried to specfiy the model training on the second GPU card in the server.
But there still allocate some GPU memory in first GPU card.
is there any way to Force the training process only using the single card memory?

BTW: the training start with python3 multiprocess is really slow.

@jhultman

Bbox conversion

Hi,

I think that the output of the model with respect to the bounding boxes seems to be [x,y,z,w,l,h,yaw] vector. I wanted to ask if this is correct and if yes is there a way to convert it to standard KITTI version which I think is [x,y,z,q.x,q.y,q.z,q.w,w,l,h]. Can you please provide some pointers on the same?

about the cls_loss and reg_loss compute

this line code in proposal.py
loss = cls_loss + self.cfg.TRAIN.LAMBDA * reg_loss
https://github.com/jhultman/PV-RCNN/blob/master/pvrcnn/detector/proposal.py#L139

why the total loss = cls_loss + 0.1*reg_loss ? which the cfg.TRAIN.LAMBDA usually set to 0.1
that not make sense...

[Question] data format support

Hello there!

I want to use your repository for the Detection of doorways in building scans. The data format of my dataset is .fls, which I can convert to a lot of other data formats like for example .e57, .las, .pcd, .ply etc. But I see that you currently only support KITTI-bin format like most of SOTA 3D Detection repositories.

Do you know by chance if it is in principle possible to convert my dataset into KITTI-bin format?
If not, do you think it is feasible to fork and extend your repository in such a way that it could also support other common point cloud formats? If yes, how hard would that be for a newbie?

Kind regards,

Daniel He

future reproduce plan for PV-RCNN refinement layer

I have tried to reproduce the propasal layer training(same as SECOND), and got some reasonable result in my own dataset, it look not bad.(the truck yaw angle looks like a little offset)

now I am working on Nuscenes dataset api and reproduce the refinement layer result in PV-RCNN with multi-class dataset.

But here still exist some problem:

multi-class training loss are not stable decline(maybe it's the personal dataset problem).
the inference part need to be refactory because you have set special score threshold for every detection class.
change pickle to json or messagepack to operate on serialize, because json is the same as pickle but faster.
Add dataset Eval scripts.

have you tried train on refinement layer and got some results that are make sense?
@jhultman

problem with spconv

Hi, I'm using pytorch1.4 + CUDA10.0 + spconv1.1 when training on the Nuscenes dataset.
But there exists some BUG in sparse conv layer when using spconv lib

have you occurred this problem with spconv 1.0? ( it looks like the error will happen sometimes)
And hope for a solution for using the newest dependent lib.

refer this : traveller59/spconv#74

how do we feed input (video or image ) and generate 3D boxes?

Yaw to rot_y conversion

Hi,

I am attempting to convert the output of the inference model to the KITTI 3D Object Detection format. I know that the current output of the model is [x, y, z, w, l, h, yaw]. Since this output is in the velodyne frame evident from the "yaw" angle, I used the KITTI velo_to_cam transform provided in the calib folder to get the corresponding output in the camera2 frame. By doing this I get accurate estimates of [x, y, z] but the rot_y estimate is way off. For example on the 4th training input of KITTI 3D Detection dataset, the output in the KITTI format in the velodyne frame looks like this,

Car -1 -1 -10 0 0 0 50 1.54 1.64 4.01 38.17 16.24 -0.65 0.00 0.34
Car -1 -1 -10 0 0 0 50 1.52 1.62 3.95 51.39 16.53 -0.53 0.00 0.31

The same output converted to camera2 frame looks like this,

Car -1 -1 -10 0 0 0 50 1.54 1.64 4.01 -15.95 2.69 38.01 0.16 0.34
Car -1 -1 -10 0 0 0 50 1.52 1.62 3.95 -16.15 2.74 51.23 0.42 0.31

The raw inference model output in the same format looks like this,

Car 1.54 1.64 4.01 38.17 14.69 -0.65 1.74
Car 1.52 1.62 3.95 51.39 15.01 -0.53 2.00

The ground truth for the same input for Car class looks like this,

Car 0.00 0 1.96 280.38 185.10 344.90 215.59 1.49 1.76 4.01 -15.71 2.16 38.26 1.57
Car 0.00 0 1.88 365.14 184.54 406.11 205.20 1.38 1.80 3.41 -15.89 2.23 51.17 1.58

As you can see, the output in the camera2 frame is very close to the ground truth in all attributes except the rot_y. Can you help me in figuring out if the output of yaw angle by the model is correct or not?

I used dummy values for the bbox, alpha, truncation and occlusion attributes.
The KITTI 3D detection format can be found here

Nvcc fatal error: unknown option 'openmp '

Hi Jhultman,

I'm using CUDA 10.1, MVSC 2019, Pytorch 1.4 run on WIndows 10.

After successfully ran step 0-1, I ran the spconv setup.py bdist_wheel. However, I got this error 'nvcc fatal error: unknown option 'openmp '. Most of the solution seems like the link below:

https://stackoverflow.com/questions/3211614/using-openmp-in-the-cuda-host-code

But, still got same error. I don't know what to do. Anyone encounter the same error? Please help me. Almost 2 days stuck here.

Thank you.

Which GPU did you use?

I tried to run 'inference.py' but i got an error; 'CUDA kernel failed : invalid device function'

and I found that this might be caused by the old version of GPU...
(I'm using GTX1080Ti now)

Could you tell me which GPU are you using for this PV-RCNN code?

Thank you.

FPS when detecting

Hello! Thanks for you work.
Can you tell me about possible (expected) frame rate when detecting object using a trained model?

only cars ahead can be detected?

Hi~
Now I can train and inference on my own dataset, it seems good but i am wondering why only cars ahead can be detected like this:

inference of PVRCNN and SECOND are both like this.
Thank you very much!

Running 'train.py'

I tried to run 'train.py' to train with KITTI dataset but I am stucking in following issue:

(pvrcnn) alien@alien-Alienware-Aurora-R7:~/PV-RCNN/pvrcnn$ python train.py
Setting up a new session...
Exception in user code:
$------------------------------------------------------------$
Traceback (most recent call last):
File "/home/alien/anaconda3/envs/pvrcnn/lib/python3.7/site-packages/urllib3/connection.py", line 157, in _new_conn
(self._dns_host, self.port), self.timeout, **extra_kw
File "/home/alien/anaconda3/envs/pvrcnn/lib/python3.7/site-packages/urllib3/util/connection.py", line 84, in create_connection
raise err
File "/home/alien/anaconda3/envs/pvrcnn/lib/python3.7/site-packages/urllib3/util/connection.py", line 74, in create_connection
sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/alien/anaconda3/envs/pvrcnn/lib/python3.7/site-packages/urllib3/connectionpool.py", line 672, in urlopen
chunked=chunked,
File "/home/alien/anaconda3/envs/pvrcnn/lib/python3.7/site-packages/urllib3/connectionpool.py", line 387, in _make_request
conn.request(method, url, **httplib_request_kw)
File "/home/alien/anaconda3/envs/pvrcnn/lib/python3.7/http/client.py", line 1252, in request
self._send_request(method, url, body, headers, encode_chunked)
File "/home/alien/anaconda3/envs/pvrcnn/lib/python3.7/http/client.py", line 1298, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "/home/alien/anaconda3/envs/pvrcnn/lib/python3.7/http/client.py", line 1247, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/home/alien/anaconda3/envs/pvrcnn/lib/python3.7/http/client.py", line 1026, in _send_output
self.send(msg)
File "/home/alien/anaconda3/envs/pvrcnn/lib/python3.7/http/client.py", line 966, in send
self.connect()
File "/home/alien/anaconda3/envs/pvrcnn/lib/python3.7/site-packages/urllib3/connection.py", line 184, in connect
conn = self._new_conn()
File "/home/alien/anaconda3/envs/pvrcnn/lib/python3.7/site-packages/urllib3/connection.py", line 169, in _new_conn
self, "Failed to establish a new connection: %s" % e
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f81a1ee1350>: Failed to establish a new connection: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/alien/anaconda3/envs/pvrcnn/lib/python3.7/site-packages/requests/adapters.py", line 449, in send
timeout=timeout
File "/home/alien/anaconda3/envs/pvrcnn/lib/python3.7/site-packages/urllib3/connectionpool.py", line 720, in urlopen
method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
File "/home/alien/anaconda3/envs/pvrcnn/lib/python3.7/site-packages/urllib3/util/retry.py", line 436, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=8097): Max retries exceeded with url: /env/main (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f81a1ee1350>: Failed to establish a new connection: [Errno 111] Connection refused'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/alien/anaconda3/envs/pvrcnn/lib/python3.7/site-packages/visdom/init.py", line 711, in _send
data=json.dumps(msg),
File "/home/alien/anaconda3/envs/pvrcnn/lib/python3.7/site-packages/visdom/init.py", line 677, in _handle_post
r = self.session.post(url, data=data)
File "/home/alien/anaconda3/envs/pvrcnn/lib/python3.7/site-packages/requests/sessions.py", line 578, in post
return self.request('POST', url, data=data, json=json, **kwargs)
File "/home/alien/anaconda3/envs/pvrcnn/lib/python3.7/site-packages/requests/sessions.py", line 530, in request
resp = self.send(prep, **send_kwargs)
File "/home/alien/anaconda3/envs/pvrcnn/lib/python3.7/site-packages/requests/sessions.py", line 643, in send
r = adapter.send(request, **kwargs)
File "/home/alien/anaconda3/envs/pvrcnn/lib/python3.7/site-packages/requests/adapters.py", line 516, in send
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=8097): Max retries exceeded with url: /env/main (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f81a1ee1350>: Failed to establish a new connection: [Errno 111] Connection refused'))
[Errno 111] Connection refused
Loading cached annotations: ../data/cache/train.pkl
Creating annotations: 100%|███████████████████████████████████████████████| 3712/3712 [00:01<00:00, 1878.93it/s]
Filtering points: 100%|████████████████████████████████████████████████████| 3712/3712 [00:08<00:00, 434.06it/s]
Caching annotations: ../data/cache/train.pkl
Building database: 100%|███████████████████████████████████████████████████| 3712/3712 [00:27<00:00, 133.68it/s]
Traceback (most recent call last):
File "train.py", line 105, in
main()
File "train.py", line 92, in main
scheduler = build_lr_scheduler(optimizer, cfg, start_epoch, len(dataloader))
File "train.py", line 79, in build_lr_scheduler
epochs=cfg.TRAIN.EPOCHS, last_epoch=last_epoch)
File "/home/alien/anaconda3/envs/pvrcnn/lib/python3.7/site-packages/torch/optim/lr_scheduler.py", line 1170, in init
super(OneCycleLR, self).init(optimizer, last_epoch)
File "/home/alien/anaconda3/envs/pvrcnn/lib/python3.7/site-packages/torch/optim/lr_scheduler.py", line 41, in init
"in param_groups[{}] when resuming an optimizer".format(i))
KeyError: "param 'initial_lr' is not specified in param_groups[0] when resuming an optimizer"

Do you recognize any problem here?

I put the KITTI dataset at the path as follows:
../data/kitti/
and the name of directories in each 'training' and 'testing' are 'calib' and 'velodyne' (not 'reduced_velodyne')

infer top K predicted boxes?

How can i get top K predicted boxes from out = net.proposal(item) in inference.py?

Evaluation

How could I evaluate the model ?
Would you be so kind to write the part of code or at least tell how it should be done?
Thank you!

What is the 'LOC function'?

Data in 'inference.py'

Actually I tried to download '000007.bin' and '000008.bin' by myself but by any chance.. do you have any script to download the KITTI dataset into pre-defined path (in the code) as follows: '../data/kitti/training/velodyne_reduced/000007.bin' ?

Can you provide pretrained weights and inference.md?

Recall of RPN stage

Hi,I wrote the eval_rpn.py based on your code, and the results of operation are as follows:
total bbox recall(thresh=0.100): 12516 / 19060 = 0.656663
total bbox recall(thresh=0.300): 12242 / 19060 = 0.642288
total bbox recall(thresh=0.500): 11379 / 19060 = 0.597009
total bbox recall(thresh=0.700): 8201 / 19060 = 0.430273
total bbox recall(thresh=0.900): 653 / 19060 = 0.034260

The recall with a threshold of 0.7 is quite different from that in the paper. Is there something wrong with the code or some evaluation details I didn't pay attention to？

eval_rpn.zip

please, how can I train the network with my own point cloud dataset?

thank you for your work.
I have my own point cloud dataset. The point cloud dataset is not from lidar.
I want to train the network with my dataset, please help me!

Some problem in TargetAssigner

I am trying to train my own lidar dataset(9 classes) on the repo.
And here seems something wrong in tensor shape mismatch which occured in ProposalTargetassigner part.

matches = torch.stack(matches).view(self.anchors.shape[:-1])
match_labels = torch.stack(match_labels).view(self.anchors.shape[:-1])

here is my log:

what's the meaning of full_idx in get_matches func?
@jhultman

Please mark it as the non-official implementation of PV-RCNN

Hi,

Thanks for your interests in our work.
Please mark it as the non-official implementation of PV-RCNN in your README in case of the unnecessary confusion for other users.

Thank you!

jhultman / vision3d Goto Github PK

vision3d's Introduction

Vision 3D

Project goals

Status

Usage

Installation

Sample results on validation data (KITTI)

Citing

Contributions

Acknowledgements and licensing

vision3d's People

Contributors

Stargazers

Watchers

Forkers

vision3d's Issues

Recommend Projects

Recommend Topics

Recommend Org