lmb-freiburg / mv3d Goto Github PK
View Code? Open in Web Editor NEWMulti-view 3D Models from Single Images with a Convolutional Network
Multi-view 3D Models from Single Images with a Convolutional Network
From what I could find on the Panda3D forums, the obj2egg
utility does not support textures/.mtl
files. So, for example, converting the ShapeNet object ShapeNetCore/02958343/114b662c64caba81bb07f8c2248e54bc
(a "NYPD Highway Patrol Dodge Charger") to a BAM file using:
obj2egg model_normalized.obj model_normalized.egg
egg2bam model_normalized.egg model_normalized.bam
and then running the following code:
from PIL import Image
from renderer import Renderer
from panda3d.core import VBase4
renderer = Renderer(False, False)
renderer.alight.setColor(VBase4(1, 1, 1, 1))
renderer.models = renderer.loader.loadModel(["model_normalized.bam"])
renderer.selectModel(0)
lights = [[3.5, 0, 60, 13]]
rad = 2.0
el = 15
az = 60
blur = 0
blending = 0
(im, dm) = renderer.renderView(
[rad, el, az], lights, blur, blending, default_bg_setting=0
)
image = Image.fromarray(im)
image.show()
produces this as output:
@treeform's forked obj2egg
tool found here seems to add support for textures, but has not been updated in almost 10 years. Can you share the exact steps you were doing to convert the objects?
Hi, after reading the paper, I have some questions
The paper used three parameters, azimuth, elevation and distance to express the labels. I assume rad is the distance. when we use both sin and cos to express the angle, is this way to make the angle unique ? why can't we express angle as angle = angle / 360 to make it between [0-1] or use sin() or cos() to express the angle ?
the code is in https://github.com/lmb-freiburg/mv3d/blob/master/utils/tf_utils.py#L159
https://github.com/lmb-freiburg/mv3d/blob/master/bg_nodm.py#L92
gt_cm = tf.slice(self.images2, [0, 0, 0, 0], [64, 128, 128, 3])
gt_sm = tf.slice(self.images2, [0, 0, 0, 3], [64, 128, 128, 1])
sm = gt_sm
gt_sm = gt_sm * 0.75
pr_cm = tf.slice(self.gen, [0, 0, 0, 0], [64, 128, 128, 3])
pr_sm = tf.slice(self.gen, [0, 0, 0, 3], [64, 128, 128, 1])
self.loss = euclidean_loss(tf.mul(gt_cm, sm), tf.mul(pr_cm, sm)) +\
0.1 * euclidean_loss(gt_sm, pr_sm)
in the first euclidean_loss(tf.mul(gt_cm, sm), tf.mul(pr_cm, sm))
, gt_cm is the ground truth in color whereas sm is the ground truth in depth, pr_cm is the predication in color , In my understanding it should be like euclidean_loss(tf.mul(pr_cm, sm), tf.mul(pr_cm, sm))
in the second euclidean_loss(gt_sm, pr_sm)
, gt_sm is the 0.75 * ground truth in color, pr_sm is the predicted depth, why these two are used in loss?
in the convolution layer, the channel are increased from 16,32,64,128,256, my question is why the lower layer has less channels (the lower layer describes more detail feature ) while the higher layers has more channels (the higher layers has more semantic information)
Thanks
Hi, I'm attempting to re-implement the code from the mv3d work as part of a different project. I am using the script utils/render_test_set.py (with modifications for locating the file paths) to render examples, but while the depth maps appear to render correctly, the rgb images render as simple black and white silhouettes. Any insight on why this may be the case? We are using OpenGL, the default rendering mode for Panda3D.
I run the nobg_nodm.py , it has something wrong:
Traceback (most recent call last):
File "nobg_nodm.py", line 191, in
net.test(0)
File "nobg_nodm.py", line 136, in test
log_value(self.writer, total_loss, 'test_loss', global_iter)
AttributeError: mv3d instance has no attribute 'writer'
I run in test mode and I downloaded the data and snapshots.What's the problem?
ValueError: Dimension 1 in both shapes must be equal, but are 4096 and 64. Shapes are [64,4096] and [64,64].
From merging shape 0 with other shapes. for 'concat/concat_dim' (op: 'Pack') with input shapes: [64,4096], [64,64].
is this because of tensorflow version, I cant install tensorflow= 0.12 on windows (which is requirement for this code) , but using tensorflow 1.14.0 currently
can anyone please help me on this
When the Renderer
is initialized with depth
set to False
, dm_unit
is never set in renderView
even though it is always returned.
Hi Team,
Can you please share the trained model for generating chairs also using mv3d and tensorflow?
need it for a project, really appreciate it, thanks
Im facing this issue, please help t resolve the same
百度base找不到,pip也一直下不下来,这是个啥库呀?
还有这个renderer.py中的第88行,render.setLight(plnp)也报错,说没有render
Hi, your work is impressive. When I read the paper, I found inconsistent fc layer output size ( 1024 in the paper vs 4096 in the code). So which one do you choose finally.
Hello, I am a novice in 3D learning. I want to know how to get training data. Thank you.
When I tried to run bg_nodm.py, it has something wrong:
ValueError: cannot reshape array of size 0 into shape (64,128,128,1)
Maybe three is something wrong : mask.astype(dtype=np.float32).reshape(
self.resolution, self.resolution, 1), im)
and
full_output = np.concatenate( (cl2, mask.reshape(self.batch_size, 128, 128, 1)), axis=3)
If I just run:
python3 nobg_nodm.py
in the root working directory, I get the follow error:
Traceback (most recent call last):
File "nobg_nodm.py", line 5, in <module>
import utils.realtime_renderer as rtr
File "/home/michael/mv3d/utils/realtime_renderer.py", line 1, in <module>
from renderer import *
ModuleNotFoundError: No module named 'renderer'
If I move everything out of utils
into the root working directory and change:
import utils.realtime_renderer as rtr
from utils.tf_utils import *
to:
import realtime_renderer as rtr
from tf_utils import *
it makes it past this error.
This commit did not fix this issue for me. Adding:
self.writer = None
to __init__
in mv3d
makes it past this error.
When I run:
python3 nobg_nodm.py
in test mode, I get the following error:
loading test set...
done.
WARNING:tensorflow:From nobg_nodm.py:86 in buildModel.: scalar_summary (from tensorflow.python.ops.logging_ops) is deprecated and will be removed after 2016-11-30.
Instructions for updating:
Please switch to tf.summary.scalar. Note that tf.summary.scalar uses the node name instead of the tag. This means that TensorFlow will automatically de-duplicate summary names based on the scope they are created in. Also, passing a tensor or list of tags to a scalar summary op is no longer supported.
loading snapshots/nobg_nodm/snapshot435000-435000...
done.
Traceback (most recent call last):
File "nobg_nodm.py", line 186, in <module>
net.test(0)
File "nobg_nodm.py", line 120, in test
self.labels: batch_labels})
File "/home/michael/.pyenv/versions/3.6.6/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 766, in run
run_metadata_ptr)
File "/home/michael/.pyenv/versions/3.6.6/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 943, in _run
% (np_val.shape, subfeed_t.name, str(subfeed_t.get_shape())))
ValueError: Cannot feed value of shape (0,) for Tensor 'input_images:0', which has shape '(64, 128, 128, 3)'
Seems like it's probably related to this. I'll see if I can figure it out.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.