Would you show how SSTNet work on the S3DIS? about sstnet HOT 11 OPEN

Hi, thank you very much for your attention to our work.
I also foresee that someone will ask S3DIS related questions, so I hope to give a unified reply to this question here.

The code of S3DIS is too messy and I do not have enough time to organize it.

I will tell you some details about the S3DIS.

1. The dataset and preprocess API can refer to gorilla3d's preprocess and gorilla3d's datasets. The voxelization parameters follow ScanNet's, you can see it in the default parameters in S3DISInstDev
2. The superpoints of S3DIS are generated by the superpoint-graph, you just need to download the pretrained model and follow their S3DIS.md, and you can get the superpoints of S3DIS.
3. In the training network setting, the S3DIS's setting is strange, it treat the cell/floor/wall as instances. And we found that most scenes just have one cell and one floor, so we will group all cell/floor points as one instance, while this operation will damage the performance slightly. And for the walls, we treat them as normal object instances.(We consider that each scene may have several walls)

I feel sorry for this. And I hope these can help you.

from sstnet.

I do appreciate for your quick reply.
Now, I have already finished the second step above. The superpoints are generated.
Since the scannetv2 is too large, I still downloading it and I try to apply s3dis directly to the network.
Here is a special question. From 146 line to 150 line in scannetv2.py, the superpoints and xyz uses the same valid_idx array. So I assume that superpoints and xyz has the same shape. But the superpoints I generate seems have a different shape. If it means I got some mistake or misunderstanding?

Here is the code:

xyz_middle = xyz_middle[valid_idxs]
xyz = xyz[valid_idxs]
rgb = rgb[valid_idxs]
semantic_label = semantic_label[valid_idxs]
superpoint = np.unique(superpoint[valid_idxs], return_inverse=True)[1]

from sstnet.

You are right. The 0-dim of superpoint ids should be equal to xyz's.

And superpoint generation in superpoint_graph will vocalize the point cloud S3DIS first, so it will downsample the number.

I guess that the superpoint_graph processing may return a point-to-voxel map, which can help you realize this.(assign the superpoint id for each point).
You can read the code in superpoint_graph partition.

I'm sorry that I forget some details about it.

from sstnet.

Thank you.
I think I get what I need, like the image shows.

from sstnet.

Thank you. I think I get what I need, like the image shows.
I’m also dealing with a dataset that is similar to S3DIS recently. Can you share your code on S3DIS datasets?

from sstnet.

Hi, Could you please share your code on S3DIS datasets? I would be grateful if you could do that for me I am in dire need of the code for S3DIS dataset. Please. Thank you in advance.

from sstnet.

Hi, Could you please share your code on S3DIS datasets? I would be grateful if you could do that for me I am in dire need of the code for S3DIS dataset. Please. Thank you in advance.

The code for S3DIS dataset has been released in gorilla3d.

from sstnet.

@lzhnb Thank you for your quick reply. However,I have got some error and raised another issue . It would be really helpful if you could help me with that issue.

from sstnet.

I do appreciate for your quick reply. Now, I have already finished the second step above. The superpoints are generated. Since the scannetv2 is too large, I still downloading it and I try to apply s3dis directly to the network. Here is a special question. From 146 line to 150 line in scannetv2.py, the superpoints and xyz uses the same valid_idx array. So I assume that superpoints and xyz has the same shape. But the superpoints I generate seems have a different shape. If it means I got some mistake or misunderstanding?

Here is the code:

xyz_middle = xyz_middle[valid_idxs] xyz = xyz[valid_idxs] rgb = rgb[valid_idxs] semantic_label = semantic_label[valid_idxs] superpoint = np.unique(superpoint[valid_idxs], return_inverse=True)[1]

@Taochen12138 How could you solve the second step? How did you generate superpoints? Please help me. Thank you in advance.

from sstnet.

@lzhnb

I do appreciate for your quick reply. Now, I have already finished the second step above. The superpoints are generated. Since the scannetv2 is too large, I still downloading it and I try to apply s3dis directly to the network. Here is a special question. From 146 line to 150 line in scannetv2.py, the superpoints and xyz uses the same valid_idx array. So I assume that superpoints and xyz has the same shape. But the superpoints I generate seems have a different shape. If it means I got some mistake or misunderstanding?

Here is the code:

xyz_middle = xyz_middle[valid_idxs] xyz = xyz[valid_idxs] rgb = rgb[valid_idxs] semantic_label = semantic_label[valid_idxs] superpoint = np.unique(superpoint[valid_idxs], return_inverse=True)[1]

Thank you. I think I get what I need, like the image shows.

Hi, how did you solve the issue as I am also getting dimension mismatch error, where it says my dimension of superpoint graph is not the same as the dimension of semantic label? It would be immense help if you could give me some suggestions. Thank you in advance.

from sstnet.