Hi, thanks or your interest in the project.

Why using grid sampling ?

Voxelization is required to mitigate memory use and smoothen potential variations in point density. Removing it may blow your memory use on very dense point clouds (eg 3DIS). As a rule of thumb, you want your voxel resolution to be about twice as high as the characteristic dimension of your smallest object of interest (think Nyquist Shannon theorem). Rather than removing the voxelization altogether, I would recommend trying to reduce the voxel size to suit your needs instead.

Is there any possibility to run inference without sampling and keep the original data points?

See issue #9. For reasons explained above, the bulk of computation should probably not operate on full resolution. You can, however, convert the superpoint-level predictions to full-resolution output. As stated in the referenced issue, I have not had time to work on implementing this. It should be quite straightforward however, just need to keep track of per-voxel point indices, store them at preprocessing, and load them at inference if full-resolution is required. Pull requests are welcome 😉

Your actual error

AssertionError: Expected Data.y to hold `(num_nodes, num_classes)` histograms, not single labels

This happens because GridSampling not only voxelizes your data, but also does some data preparation. In particular, it converts integer point labels to voxel label histograms. To avoid this error, I would recommend keeping GridSampling but reducing the voxel size to your needs. You can make as small as you want, if you want to operate on your full resolution point cloud, though you may run into downstream memory issues...

from superpoint_transformer.

Thanks for your answers, I get it now.
For the full-resolution generalization, is it possible to do this using SaveNodeIndex class to save the indices?
I can work on that and make a pull request if I get good results.

@akharroubi

from superpoint_transformer.

Yes, but not only. Off the top of my head, to implement this feature, one would need to do the following:

• call SaveNodeIndex to store full-res point indices in a Data attribute full_resolution_idx (for instance) before calling GridSampling
• modify GridSampling to store the full_resolution_idx of the points in each voxel, in the full_resolution_idx attribute of the output voxelized Data. The Cluster structure is adapted for holding this type of data efficiently
• save the full_resolution_idx at the end of preprocessing
• at training time, do not load full_resolution_idx (slow and useless)
• at inference time, optionally load full_resolution_idx if required by a new boolean full_resolution parameter (need to get familiar with configs structure and NAG and Data loading process)
• if full_resolution is True, compute the full-resolution output by simply distributing the per-superpoint semantic predictions to voxels, and then to full-res points

Some steps (saving / loading) will require a good comprehension of the project structure.

from superpoint_transformer.

Closing this issue for now, which I consider a duplicate of the feature request in : #9

Feel free to re-open it if you work on implementing the above-described pipeline.

PS: if you are interested in this project, don't forget to give it a ⭐, it matters to us !

from superpoint_transformer.