chenhsuanlin / 3d-point-cloud-generation Goto Github PK

Thank you for providing your code and data. But there seems to be some error inside the data.
I had download the data from here. But when I run tar -zxf 3D-PCG-data.tar.gz it shows the error as following.
gzip: stdin: invalid compressed data--format violated tar: Unexpected EOF in archive tar: Unexpected EOF in archive tar: Error is not recoverable: exiting now
I also tried tar zxvf 3D-PCG-data.tar.gz, the same error message is shown.

Hello! I've read your paper and have a keen interest on your work. However, I get some problem at the first step. I can't download your dataset by wget or any other way I have. I suspect that may your link is not available now. Would you please give me a new link or any other suggestion to make it? Thank you.

I couldn't find https://cmu.box.com/shared/static/s4lkm5ej7sh4px72vesr17b1gxam4hgy.gz .
https://cmu.box.com has been changed to https://community.box.com.
Could you update the dataset link? Thank you very much!

Thanks for providing the code. I need to ask you some questions. 1, Can the model conduct the image of size 1800*1800? 2, Can I replace this data with my own data, because I can't download your data? 3，Can I add the image registration process (to get a large viewpoint image) after obtaining the 2D images? Thanks for your attention!

Best regrads,

Tingting Dan

Hi,

I've noticed that both render.py and render_fixed.py try to access a file called 'model_normalized.obj', which is supposed to be under the 'models' directory for each object code in the specified series (for example: '/dataset/ShapeNetCore.v1/03001627/1006be65e7bc937e9141f9b58470d646/models/model_normalized.obj')
But it seems that such a file or directory doesn't exist.

I downloaded the image zip file for the chair synset ( synset id: 03001627) under the ShapeNetCore v1 dataset from shapenet.org.

I'd really like to know how model_normalized.obj is created. Is it supposed to be available in the zip file? Or have I missed out any steps in between scripts/evaluate_dist.sh and ./run.sh 03001627 8?

Any help is highly appreciated. Thanks in advance!

Hi @chenhsuanlin ,

Thanks for sharing code and dataset!!
But according to your description on dataset and rendering depth images, it seems that RGB images, testGT of shapenet models (except chairs) and Train/test split files(also except chairs) are not given. Could you please release your multi-category dataset or just release your method for getting the inputRGB images and testGT (point cloud)?

Thanks for your sharing and looking forward to your reply sincerely.

Sorry if it's a silly question: is there any way to convert the outputs (Z an trans) of the pointcloud to a PLY or OBJ file?

I think it can be useful for non experts like me who just want to use the code out-of-box for experiments.

While training with mask loss, loss is defined on the projected masks obtained using point cloud. I would like to understand how the point cloud is updated based on this loss. Assuming that for a given pixel, GT projection is 0 and predicted projection is 1, will only those 3D points which contributed to that pixel get non-zero gradients ? If yes, which points get non-zero gradients when GT for the pixel is 1 and prediction is 0 ?

Hi,

I tried to train the model by following the instruction.
But when I tried to fine-tuning with joint 2D optimization using the instruction scripts/run-finetune.sh.
However, during training, the memory usage increase rapidly. Is this a normal case? Thank you.

I used your model to train and test it works great, but currently I am able to understand .mat file output.
as per your code you are using scipy to save in mat with image and cloud points but how I able to convert to exr for viewing in the blender?

Great work by the way.

Hi! I just ran your code with the weights you provided by evaluate.py script on the test list provided with the dataset. I can see corresponding .mat files created in output directory. beg your pardon as I have no experience in 3D. I want to know the input origin which give 24,64,64,3 shape and so does the output mat file generated when I read them. How exactly can I visualize the 3D point clouds generated and how would I know what image am I providing the model as they are bunch of npy files.

In transform.py, Line 57, x and y co-ordinates are being converted to int datatype. Since the operation is not differentiable, are gradients for mask loss wrt to point cloud xyz defined ?

Very great work!
I want to ask, is your RGB images rendered by yourself? Is there color information?
If I want to get a dataset which includes RGB images and point clouds of each single object, is there such a data set?
Looking forward to your reply!
Thank you!

Hello,

I want to render a depth map. I've already had a 3D point-cloud model and a set of camera poses (quaternion + translation vector). Can I make a change in your code to run with my dataset?
Hope to hear from you. Thank you so much.

Best,

Hi @chenhsuanlin ,

Thanks for providing your code. Could you please share your trained weights on your single-category and multi-category experiments?

Thanks a lot!

Hi, thank for the great work.

I would like to understand more about the differentiable transforms functions in transforms.py
As I don't have background in 3D-2D transform matrix, could you give me some helpful links to understand more about it?

Specifically, I would like to know more about HomMatrix as in transParamsToHomMatrix() function.
Any intro links or correct keywords for Google is appreciated.

Hi @chenhsuanlin

Could you please share the weights?
I was not able to reproduce your results using the code from this repo.
Following the steps from "Running the code" from the readme I get worse performance i.e. GT->Pred is around 20.

Looking forward to hearing from you.

The code uses both the depth and depth_fixed8 matrices (that already come pre-rendered in the downloaded data/ folder) to generate the 3D point cloud of an specific inputRGB image. Could you please share how you created those matrices so that we can do it for our own 2D images?

Thank you in advance, hope it's not a problem