rehg-lab / 3dshapegen Goto Github PK
View Code? Open in Web Editor NEWCode for 3D Reconstruction of Novel Object Shapes from Single Images paper
License: MIT License
Code for 3D Reconstruction of Novel Object Shapes from Single Images paper
License: MIT License
Hi
I was wondering if you could enable link sharing so that the data can be downloaded from a Linux console with wget?
Thank you
Hi,
Thanks a lot for releasing the amazing codes. Everything works well except for this small rendering issue.
We would like to make use of your rendering codes. However, I cannot get exactly the same images when I run wrapper.py with the default data_generation_parameters.json (my blender version is 2.79). The scale of the object is smaller than the provided rendered images. Here is an example. The first image is what I obtained by running wrapper.py and the second image is provided in your repo (ShapeNet55_3DOF-VC_LRBg). In addition, current codes do not seem to support blending the background. Can you also released the background part? Thank you very much!
Hi,
thanks for providing the code to your paper.
I wanted to generate meshes from the sdf values using your pipeline eval.py
but run into the error:
(sdf_net)` user@host SDFNet: python eval.py
Loading model from 3DShapeGen/out/train_overfit_test/best_model_iou.pth.tar
Loading data...
0%| | 0/1 [00:00<?, ?it/s]
./isosurface/computeMarchingCubes: error while loading shared libraries: libpng12.so.0: cannot open shared object file: No such file or directory
in the environment.yaml another libpng version is specified: - libpng=1.6.37=hed695b0_0
I tried to downgrade the libpng version to 12 in the conda environment. This did not work.
Could you please provide (or point me to) the codebase s.t. I can build MarchingCubes myself with libpng16 or could you update this repo with a new build that uses the version from the environment?
Thanks a lot in advance.
Hi all,
Thank you for releasing the code for the paper! Could you please add the license information?
Don't know if you can choose, but it would be great if the license was MIT.
Thanks for sharing the code and data.
The SDFNet README listed some data. I got a "permission denied" message when accessing "3-DOF VC D+N Pred" at this link:
https://www.dropbox.com/s/qz227pr2vbrzu9l/ShapeNet55_HVC_LR_PRED.tar
Other data links seem to work fine.
BTW, is "3-DOF VC D+N Pred" the data output from the Stage 1 (2D -> 2.5D) network after processing the ground truth data in the link "3-DOF viewpoint LRBg ShapeNetCore.v2 renders"?
Hi,
Thanks again for this work and the awesome code / data release! We will be trying to compare with your tables so here a few queries to make sure we are doing things properly.
Our main source of hesitations comes from :
Here is my current understanding please correct me if you spot something wrong :
Fig 1 : SDFNet trained with estimated depth
Fig 2 : SDFNet trained with estimated depth
Fig 3 : SDFNet trained with ground-truth depth
Table 2 : GenRe data. 3 categories for training / 10 for testing , using estimated depth to train/test
Table 3 : ShapeNet data. 13 categories for training / 42 for testing , renderings with 2 DoF, using estimated depth to train/test
Table 4 : ShapeNet data. 13 categories for training / 42 for testing , renderings with 3 DoF, using estimated depth to train/test
Table 5 : ShapeNet data. 13 categories for training / 42 for testing , using ground-truth depth to train/test
Table 6 : ShapeNet data. 13 categories for training and testing , using estimated and ground-truth depth to train/test
Some questions to make sure I understand :
Thank you so much for your precious help,
Thibault
Hi, thanks for your release. I tried to get 2-DOF_VC_with predicted_Depth+Normals by their link when I found on READEME.md, but I don't have access. Can you help me with this?
Hi, there.
Thanks for your release. I'm interested in your work and trying to implement it. I can get reasonable results directly using images without depth and normal estimators. But it doesn't work when depth and normal estimators are also implemented. I want to make sure if the two parts are trained totally separated? And do you add additional operations on the depth maps, like the sigmoid or clamp function? Because I also found some values less than 0.
This is my code of depth and normal estimators:
class Net(nn.Module):
def __init__(self, out_planes, layer_names, input_planes=3):
super().__init__()
# Encoder
module_list = list()
resnet = resnet18(pretrained=True)
in_conv = nn.Conv2d(input_planes, 64, kernel_size=7, stride=2, padding=3, bias=False)
module_list.append(
nn.Sequential(
resnet.conv1 if input_planes == 3 else in_conv,
resnet.bn1,
resnet.relu,
resnet.maxpool
)
)
module_list.append(resnet.layer1)
module_list.append(resnet.layer2)
module_list.append(resnet.layer3)
module_list.append(resnet.layer4)
self.encoder = nn.ModuleList(module_list)
# Decoder
for out_plane, layer_name in zip(out_planes, layer_names):
module_list = list()
revresnet = revuresnet18(out_planes=out_plane)
module_list.append(revresnet.layer1)
module_list.append(revresnet.layer2)
module_list.append(revresnet.layer3)
module_list.append(revresnet.layer4)
module_list.append(
nn.Sequential(
revresnet.deconv1,
revresnet.bn1,
revresnet.relu,
revresnet.deconv2
)
)
module_list = nn.ModuleList(module_list).cuda()
setattr(self, 'decoder_' + layer_name, module_list)
def forward(self, im):
# Encode
feat = im
feat_maps = list()
for f in self.encoder:
feat = f(feat)
feat_maps.append(feat)
x = feat_maps[-1]
for idx, f in enumerate(self.decoder_depth):
x = f(x)
if idx < len(self.decoder_depth) - 1:
feat_map = feat_maps[-(idx + 2)]
assert feat_map.shape[2:4] == x.shape[2:4]
x = torch.cat((x, feat_map), dim=1)
depth_output = x
x = feat_maps[-1]
for idx, f in enumerate(self.decoder_mask):
x = f(x)
if idx < len(self.decoder_depth) - 1:
feat_map = feat_maps[-(idx + 2)]
assert feat_map.shape[2:4] == x.shape[2:4]
x = torch.cat((x, feat_map), dim=1)
mask_output = x
x = feat_maps[-1]
for idx, f in enumerate(self.decoder_normal):
x = f(x)
if idx < len(self.decoder_depth) - 1:
feat_map = feat_maps[-(idx + 2)]
assert feat_map.shape[2:4] == x.shape[2:4]
x = torch.cat((x, feat_map), dim=1)
normal_output = x
return depth_output, mask_output, normal_output
For inference:
depth, mask, normal = depth_normal_model(img_input)
depth = depth*mask
input_2 = torch.cat([depth, normal], dim=1)
sdf = model(points_input, input_2)
Hi,
Thanks for your great work and for sharing all the code to generate the data.
I am trying to get the rotation matrix from the metadata. I've been struggling a bit with this so I tried reusing the rendering script you provide i.e generate.py
to render a shapenet object with simple parameters. The script gives several errors on Mac, hence my question :
These errors are for instance :
File "utils/render_utils.py", line 66, in make_area_lamp
nodes = lamp.node_tree.nodes
AttributeError: 'NoneType' object has no attribute 'nodes'
This can be solved by setting :
lamp.use_nodes = True
but there are other errors after this.
If you have any pointers to go from the metadata files to the rotation matrix, that would be great. I tried inversing the sequence of Euler-angle transformations in generate.py
but the resulting matrix is not consistent with the renderings so far.
Cheers,
Thibault
Where can I find the data split and rendering meta data of ABC dataset?
Hi,
Thank you for this work. I am interested in using the image only version as a baseline. Before I try training it myself, I was wondering if you happened to have the pre-trained model (image only, 3DOF, VC) hanging around somewhere. It would be very much appreciated!
Thank you,
Marissa
Thank you for sharing the code! I have an issue with the depth map generation. It seems that the depth map rendered with cycles engine in blender 2.79 is not the z-value, which is the orthogonal distance. https://blender.stackexchange.com/questions/87504/blender-cycles-z-coordinate-vs-camera-centered-z-coordinate-orthogonal-distan
Will this depth map behavior affects the GenRe results? I think that the depth map used by GenRe rendered by Mitsuba is keeping the z-value.
It seems that the depth estimation part is missing in the repo. Is it possible to open-source depth estimation codes as well? Thanks!
Hi,
I am trying to train and evaluate the network for a small scale experiment. I have managed to get everything working up to training— setting up environment, rendering input image data, changing configurations, managing dependencies issues...
At the start of training in train.py
I get the following error:
RuntimeError: cuda runtime error (8) : invalid device function at /opt/conda/conda-bld/pytorch_1544174967633/work/aten/src/THC/THCGeneral.cpp:405
I am running the training on a single NVIDIA GeForce GPU with 25GB memory usage. Driver Version: 510.60.02 and CUDA Version: 11.6. I suspect that this error comes from some incompatibility between my GPU CUDA version and the PyTorch and other packages versions set up by the environment provided on environment.yml
I was wondering if anyone or the authors have run into a similar issue or have any suggestions on how to manage this possible incompatibility.
I tried to install higher versions of PyTorch, torchvision, and install cudatoolkit on the environment but this lead to incompatibilities with other packages' versions specified in environment.yml
. conda update --all
also created dependency issues
Thank you very much for releasing the nice codes. I would like to reproduce your results reported in Table 3 in your arxiv paper. Is it possible to release the pretrained model (SDFNet) and predicted depth and normal used for reproducing Table 3? Thank you very much!
Hi again,
I have some general questions about the repo:
How many GPU's, GPU memory and RAM do you use to train the network?
device_ids=[0]
)In your paper the NW architecture contains a U-ResNet that predicts depth, normals and silhouette from a RGB image. Where can I find this in your repo? The SDFNet/model.py only contains encoder and 3D decoder.
GenRe-ShapeHD/networks/
Why do you need 2 different marching cubes algorithms, i.e. libmcubes for occupancy representation and the marching cubes in SDFNet/isosurface?
Best regards, Jan
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.