leofansq / scf-net Goto Github PK

[CVPR2021] SCF-Net: Learning Spatial Contextual Features for Large-Scale Point Cloud Segmentation

License: MIT License

Python 37.88% Shell 1.70% C++ 58.45% C 0.38% Cython 1.60%

point-cloud semantic-segmentation autonomous-driving scene-understanding

scf-net's Introduction

Hi there 👋

I'm a Researcher at Institute for AI Industry Research, Tsinghua University (AIR, THU). I obtained my M.S. degree in Automation from Institute of Automation, Chinese Academy of Sciences (CASIA) in 2022, and the B.E. degree in Automation from Shanghai Jiao Tong University (SJTU) in 2019.

🔭 I'm currently working on Perceptual Intelligence and Cognitive Intelligence
📖 Check my publications via google scholar
⚡ Some of my projects are opensourced at GitHub
Intern Opening (Research Intern @ AIR): Kindly email your resume and availability to [email protected], I will reach back to you if you are a good fit.

Some Tools

Some Researches

scf-net's People

Contributors

Stargazers

Watchers

Forkers

liuxinren456852 y-sheng kinggreat24 yeminglang yanlingz yellowpuppy whuhxb liangzhendong123 xingzhewujiang1 jordanesikati

scf-net's Issues

d_out

Hi Siqi,

Thank you for release your work.

I am tring to use your LFA in my network but am confused about some code.

The dimension of f_lc, shortcut, and f_gc is d_out2, so makes the returned dimension be d_out4 in this case. But in RandLA-Net, the output dimension should be d_out*2. Are they different in the architecture? Or should the dimensions of each layer be the same in your network and RandLA-Net?

Appreciate if you could help me with this problem.

in SCFNet.py:
line 276-285

    # MLP 2
    f_lc = helper_tf_util.conv2d(f_lc, d_out * 2, [1, 1], name + 'mlp2', [1, 1], 'VALID', True, is_training, activation_fn=None)
    # MLP Shotcut
    shortcut = helper_tf_util.conv2d(feature, d_out * 2, [1, 1], name + 'shortcut', [1, 1], 'VALID', activation_fn=None, bn=True, is_training=is_training)
    
    # Global Contextual Features
    f_gc = tf.expand_dims(tf.concat([xyz, lg_volume_ratio], axis=-1), -2)
    f_gc = helper_tf_util.conv2d(f_gc, d_out * 2, [1, 1], name + 'lg', [1, 1], 'VALID', activation_fn=None, bn=True, is_training=is_training)

    return tf.nn.leaky_relu(tf.concat([f_lc + shortcut, f_gc], axis=-1))

model.summary() or model parameters calculation issue

I want to Print model summary, but when I call model.summary() in s3dis.main after model = Network(dataset, cfg) attribute error occurs.
If you know any other way to calculate the number of model parameters that will be also helpful.

thankyou

Error: AttributeError: 'Network' object has no attribute 'summary'

After training and testing, the ply file generated by the test cannot be opened

Excuse me, is the ply file in the generated test/s3dis_preds containing point cloud x, y, z coordinates? Why can't the resulting ply file open? And is your point cloud forecast results can send me a copy? My mailbox is [email protected], thank you!

OOM when testing

Hello, an OOM error occured during testing after several epochs. It seems that the input is too huge ([262144,64,16]). Has this problem ever happened to you? My GPU is 1080Ti with 11GB memory.

Epoch   7, step  18. min possibility = 3.4
Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/tensorflow_core/python/client/session.py", line 1365, in _do_call
    return fn(*args)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow_core/python/client/session.py", line 1350, in _run_fn
    target_list, run_metadata)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow_core/python/client/session.py", line 1443, in _call_tf_sessionrun
    run_metadata)
tensorflow.python.framework.errors_impl.ResourceExhaustedError: 2 root error(s) found.
  (0) Resource exhausted: OOM when allocating tensor with shape[262144,64,16] and type float on /job:localhost/replica:0/task:0/device:GPU:0 by allocator GPU_0_bfc
	 [[{{node layers/Softmax_2}}]]
Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info.

	 [[Softmax/_1397]]
Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info.

  (1) Resource exhausted: OOM when allocating tensor with shape[262144,64,16] and type float on /job:localhost/replica:0/task:0/device:GPU:0 by allocator GPU_0_bfc
	 [[{{node layers/Softmax_2}}]]
Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info.

0 successful operations.
0 derived errors ignored.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/SCF-Net/semantic3d_main.py", line 404, in <module>
    tester.test(model, dataset)
  File "/home/SCF-Net/semantic3d_test.py", line 76, in test
    stacked_probs, stacked_labels, point_idx, cloud_idx = self.sess.run(ops, {model.is_training: False})
  File "/usr/local/lib/python3.5/dist-packages/tensorflow_core/python/client/session.py", line 956, in run
    run_metadata_ptr)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow_core/python/client/session.py", line 1180, in _run
    feed_dict_tensor, options, run_metadata)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow_core/python/client/session.py", line 1359, in _do_run
    run_metadata)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow_core/python/client/session.py", line 1384, in _do_call
    raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.ResourceExhaustedError: 2 root error(s) found.
  (0) Resource exhausted: OOM when allocating tensor with shape[262144,64,16] and type float on /job:localhost/replica:0/task:0/device:GPU:0 by allocator GPU_0_bfc
	 [[node layers/Softmax_2 (defined at usr/local/lib/python3.5/dist-packages/tensorflow_core/python/framework/ops.py:1748) ]]
Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info.

	 [[Softmax/_1397]]
Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info.

  (1) Resource exhausted: OOM when allocating tensor with shape[262144,64,16] and type float on /job:localhost/replica:0/task:0/device:GPU:0 by allocator GPU_0_bfc
	 [[node layers/Softmax_2 (defined at usr/local/lib/python3.5/dist-packages/tensorflow_core/python/framework/ops.py:1748) ]]
Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info.

0 successful operations.
0 derived errors ignored.

Original stack trace for 'layers/Softmax_2':
  File "home/SCF-Net/semantic3d_main.py", line 392, in <module>
    model = Network(dataset, cfg)
  File "home/SCF-Net/SCFNet.py", line 56, in __init__
    self.logits = self.inference(self.inputs, self.is_training)
  File "home/SCF-Net/SCFNet.py", line 117, in inference
    f_encoder_i = self.scf_module(feature, inputs['xyz'][i], inputs['neigh_idx'][i], d_out[i], 'Encoder_layer_' + str(i), is_training)
  File "home/SCF-Net/SCFNet.py", line 275, in scf_module
    f_lc, lg_volume_ratio = self.local_context_learning(xyz, f_pc, neigh_idx, d_out, name + 'LFA', is_training)
  File "home/SCF-Net/SCFNet.py", line 300, in local_context_learning
    f_lc = self.dualdis_att_pool(f_concat, f_dis, g_dis, d_out // 2, name + 'dis_att_pooling_1', is_training)
  File "home/SCF-Net/SCFNet.py", line 442, in dualdis_att_pool
    att_scores = tf.nn.softmax(att_activation, axis=1)
  File "usr/local/lib/python3.5/dist-packages/tensorflow_core/python/util/deprecation.py", line 507, in new_func
    return func(*args, **kwargs)
  File "usr/local/lib/python3.5/dist-packages/tensorflow_core/python/ops/nn_ops.py", line 2958, in softmax
    return _softmax(logits, gen_nn_ops.softmax, axis, name)
  File "usr/local/lib/python3.5/dist-packages/tensorflow_core/python/ops/nn_ops.py", line 2920, in _softmax
    output = compute_op(logits)
  File "usr/local/lib/python3.5/dist-packages/tensorflow_core/python/ops/gen_nn_ops.py", line 11376, in softmax
    "Softmax", logits=logits, name=name)
  File "usr/local/lib/python3.5/dist-packages/tensorflow_core/python/framework/op_def_library.py", line 794, in _apply_op_helper
    op_def=op_def)
  File "usr/local/lib/python3.5/dist-packages/tensorflow_core/python/util/deprecation.py", line 507, in new_func
    return func(*args, **kwargs)
  File "usr/local/lib/python3.5/dist-packages/tensorflow_core/python/framework/ops.py", line 3357, in create_op
    attrs, op_def, compute_device)
  File "usr/local/lib/python3.5/dist-packages/tensorflow_core/python/framework/ops.py", line 3426, in _create_op_internal
    op_def=op_def)
  File "usr/local/lib/python3.5/dist-packages/tensorflow_core/python/framework/ops.py", line 1748, in __init__
    self._traceback = tf_stack.extract_stack()

Segmentation fault

How to solve this? Thanks a lot.

2023-02-14 12:59:47.938028: E tensorflow/stream_executor/cuda/cuda_dnn.cc:343] Loaded runtime CuDNN library: 7.1.4 but source was compiled with: 7.2.1. CuDNN library major and minor version needs to match or have higher minor version in case of CuDNN 7.0 or later version. If using a binary install, upgrade your CuDNN library. If building from sources, make sure the library loaded at runtime is compatible with the version specified during compile configuration.
Area_4_office_13_KDTree.pkl 1.3 MB loaded in 0.0ssrun: error: node09: task 0: Segmentation fault (core dumped)

OA mAcc and mIoU？

Is eval accuracy and OA a concept？

关于Semantic3D测试结果的问题

注册账号之后只收到这个，没有办法激活，请问您是怎样激活账号的呢？

About visualization

Thank you for share your work.I have followed the instruction and gained the result but I met some question in visualization steps,I don't know what the label path is,can you give me some advice?Thanks a lot.

How long is the training?

What is the GPU you are using, and how long did it run on each of the two data sets?

Error while running semantic3d in test mode

Thanks for sharing your code of SCF-Net. I tried to run the training and testing of Semantic3D. While training runs smooth and some training snapshoots were created, I can't test the created models. I also downloaded your pretrained model. Still I always get the following error:

(scf) user@linux:SCF-Net> python semantic3d_main.py --mode test --gpu 1 --model_path='/daten/SCF-Net/data/semantic3d.data-00000-of-00001' 
Load_pc_0: bildstein_station1_xyz_intensity_rgb
Load_pc_1: bildstein_station5_xyz_intensity_rgb
Load_pc_2: domfountain_station1_xyz_intensity_rgb
Load_pc_3: domfountain_station2_xyz_intensity_rgb
Load_pc_4: domfountain_station3_xyz_intensity_rgb
Load_pc_5: sg27_station1_intensity_rgb
Load_pc_6: sg27_station2_intensity_rgb
Load_pc_7: sg27_station5_intensity_rgb
Load_pc_8: sg27_station9_intensity_rgb
Load_pc_9: sg28_station4_intensity_rgb
Load_pc_10: untermaederbrunnen_station1_xyz_intensity_rgb
Load_pc_11: untermaederbrunnen_station3_xyz_intensity_rgb
Load_pc_12: bildstein_station3_xyz_intensity_rgb
Load_pc_13: sg27_station4_intensity_rgb
Load_pc_14: #neugasse_station1_xyz_intensity_rgb
Load_pc_15: MarketplaceFeldkirch_Station4_rgb_intensity-reduced
Load_pc_16: StGallenCathedral_station6_rgb_intensity-reduced
Load_pc_17: birdfountain_station1_xyz_intensity_rgb
Load_pc_18: castleblatten_station1_intensity_rgb
Load_pc_19: castleblatten_station5_xyz_intensity_rgb
Load_pc_20: marketplacefeldkirch_station1_intensity_rgb
Load_pc_21: marketplacefeldkirch_station4_intensity_rgb
Load_pc_22: marketplacefeldkirch_station7_intensity_rgb
Load_pc_23: sg27_station10_intensity_rgb
Load_pc_24: sg27_station10_rgb_intensity-reduced
Load_pc_25: sg27_station3_intensity_rgb
Load_pc_26: sg27_station6_intensity_rgb
Load_pc_27: sg27_station8_intensity_rgb
Load_pc_28: sg28_Station2_rgb_intensity-reduced
Load_pc_29: sg28_station2_intensity_rgb
Load_pc_30: sg28_station5_xyz_intensity_rgb
Load_pc_31: stgallencathedral_station1_intensity_rgb
Load_pc_32: stgallencathedral_station3_intensity_rgb
Load_pc_33: stgallencathedral_station6_intensity_rgb

Preparing reprojection indices for validation and test
finished
Initiating input pipelines
WARNING:tensorflow:From /daten/SCF-Net/SCFNet.py:262: softmax_cross_entropy_with_logits (from tensorflow.python.ops.nn_ops) is deprecated and will be removed in a future version.
Instructions for updating:

Future major versions of TensorFlow will allow gradients to flow
into the labels input on backprop by default.

See `tf.nn.softmax_cross_entropy_with_logits_v2`.

/home/user/miniconda/envs/scf/lib/python3.6/site-packages/tensorflow/python/ops/gradients_impl.py:108: UserWarning: Converting sparse IndexedSlices to a dense Tensor of unknown shape. This may consume a large amount of memory.
  "Converting sparse IndexedSlices to a dense Tensor of unknown shape. "
Traceback (most recent call last):
  File "/home/user/miniconda/envs/scf/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1292, in _do_call
    return fn(*args)
  File "/home/user/miniconda/envs/scf/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1277, in _run_fn
    options, feed_dict, fetch_list, target_list, run_metadata)
  File "/home/user/miniconda/envs/scf/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1367, in _call_tf_sessionrun
    run_metadata)
tensorflow.python.framework.errors_impl.DataLossError: Unable to open table file /daten/SCF-Net/data/semantic3d.data-00000-of-00001: Data loss: not an sstable (bad magic number): perhaps your file is in a different file format and you need to use a different restore operator?
         [[{{node save_1/RestoreV2}} = RestoreV2[dtypes=[DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, ..., DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT], _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_save_1/Const_0_0, save_1/RestoreV2/tensor_names, save_1/RestoreV2/shape_and_slices)]]
         [[{{node save_1/RestoreV2/_157}} = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device_incarnation=1, tensor_name="edge_230_save_1/RestoreV2", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:GPU:0"]()]]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "semantic3d_main.py", line 403, in <module>
    tester = ModelTester(model, dataset, restore_snap=chosen_snap)
  File "/daten/SCF-Net/semantic3d_test.py", line 33, in __init__
    self.saver.restore(self.sess, restore_snap)
  File "/home/user/miniconda/envs/scf/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 1538, in restore
    {self.saver_def.filename_tensor_name: save_path})
  File "/home/user/miniconda/envs/scf/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 887, in run
    run_metadata_ptr)
  File "/home/user/miniconda/envs/scf/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1110, in _run
    feed_dict_tensor, options, run_metadata)
  File "/home/user/miniconda/envs/scf/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1286, in _do_run
    run_metadata)
  File "/home/user/miniconda/envs/scf/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1308, in _do_call
    raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.DataLossError: Unable to open table file /daten/SCF-Net/data/semantic3d.data-00000-of-00001: Data loss: not an sstable (bad magic number): perhaps your file is in a different file format and you need to use a different restore operator?
         [[{{node save_1/RestoreV2}} = RestoreV2[dtypes=[DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, ..., DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT], _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_save_1/Const_0_0, save_1/RestoreV2/tensor_names, save_1/RestoreV2/shape_and_slices)]]
         [[{{node save_1/RestoreV2/_157}} = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device_incarnation=1, tensor_name="edge_230_save_1/RestoreV2", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:GPU:0"]()]]

Caused by op 'save_1/RestoreV2', defined at:
  File "semantic3d_main.py", line 403, in <module>
    tester = ModelTester(model, dataset, restore_snap=chosen_snap)
  File "/daten/SCF-Net/semantic3d_test.py", line 20, in __init__
    self.saver = tf.train.Saver(my_vars, max_to_keep=100)
  File "/home/user/miniconda/envs/scf/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 1094, in __init__
    self.build()
  File "/home/user/miniconda/envs/scf/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 1106, in build
    self._build(self._filename, build_save=True, build_restore=True)
  File "/home/user/miniconda/envs/scf/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 1143, in _build
    build_save=build_save, build_restore=build_restore)
  File "/home/user/miniconda/envs/scf/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 787, in _build_internal
    restore_sequentially, reshape)
  File "/home/user/miniconda/envs/scf/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 406, in _AddRestoreOps
    restore_sequentially)
  File "/home/user/miniconda/envs/scf/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 854, in bulk_restore
    return io_ops.restore_v2(filename_tensor, names, slices, dtypes)
  File "/home/user/miniconda/envs/scf/lib/python3.6/site-packages/tensorflow/python/ops/gen_io_ops.py", line 1466, in restore_v2
    shape_and_slices=shape_and_slices, dtypes=dtypes, name=name)
  File "/home/user/miniconda/envs/scf/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper
    op_def=op_def)
  File "/home/user/miniconda/envs/scf/lib/python3.6/site-packages/tensorflow/python/util/deprecation.py", line 488, in new_func
    return func(*args, **kwargs)
  File "/home/user/miniconda/envs/scf/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 3272, in create_op
    op_def=op_def)
  File "/home/user/miniconda/envs/scf/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1768, in __init__
    self._traceback = tf_stack.extract_stack()

DataLossError (see above for traceback): Unable to open table file /daten/SCF-Net/data/semantic3d.data-00000-of-00001: Data loss: not an sstable (bad magic number): perhaps your file is in a different file format and you need to use a different restore operator?
         [[{{node save_1/RestoreV2}} = RestoreV2[dtypes=[DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, ..., DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT], _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_save_1/Const_0_0, save_1/RestoreV2/tensor_names, save_1/RestoreV2/shape_and_slices)]]
         [[{{node save_1/RestoreV2/_157}} = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device_incarnation=1, tensor_name="edge_230_save_1/RestoreV2", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:GPU:0"]()]]

As you may see, I had to use Python 3.6 as I couldn't install a valid environment with Anaconda meeting the other requirements (Tensorflow 1.11, Cuda Toolkit 9.0 and cudnn v7). I also tried semantic3d.index and semantic3d.meta as --model_path. Still, the model can't be read:

DataLossError (see above for traceback): Unable to open table file /daten/SCF-Net/data/semantic3d.data-00000-of-00001: Data loss: not an sstable (bad magic number): perhaps your file is in a different file format and you need to use a different restore operator?

Do have any ideas on this problem? I would appreciate any help.

Best regards
Jens

Translation to Pytorch ?

Hi, thanks for your great work ! Are you planning to make a Pytorch version of SCF Module soon ? That would be a great feature for this repo
Thank you in advance

solved

The result of L_out is nan, and Acc is 0.00

EPOCH 0
Step 00000050 L_out= nan Acc=0.00 --- 1006.39 ms/batch
Step 00000100 L_out= nan Acc=0.00 --- 924.97 ms/batch
Step 00000150 L_out= nan Acc=0.00 --- 908.08 ms/batch
Step 00000200 L_out= nan Acc=0.00 --- 760.66 ms/batch
Step 00000250 L_out= nan Acc=0.00 --- 811.27 ms/batch
Step 00000300 L_out= nan Acc=0.00 --- 967.48 ms/batch
Step 00000350 L_out= nan Acc=0.00 --- 719.22 ms/batch
Step 00000400 L_out= nan Acc=0.00 --- 828.67 ms/batch
Step 00000450 L_out= nan Acc=0.00 --- 696.24 ms/batch
Step 00000500 L_out= nan Acc=0.00 --- 873.87 ms/batch
Step 00000550 L_out= nan Acc=0.00 --- 719.05 ms/batch
Step 00000600 L_out= nan Acc=0.00 --- 788.21 ms/batch
Step 00000650 L_out= nan Acc=0.00 --- 798.09 ms/batch
Step 00000700 L_out= nan Acc=0.00 --- 925.55 ms/batch
Step 00000750 L_out= nan Acc=0.00 --- 741.91 ms/batch
0 / 200
50 / 200
100 / 200
150 / 200
eval accuracy: 0.18944735107421876
mean IOU:0.01457287315955529
Mean IoU = 1.5%
1.46 | 18.94 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00

Best m_IoU is: 1.457

visualization.py

Regarding the command of visualization“python visualization.py --dataset 'XXX' --ply_path "path_to_origin_ply" --label_path "path_to_label"”, Only need to modify the “dataset‘XXX’”？