Is it possible to export to an mtl file? about shap-e HOT 6 OPEN

I have a basic alternative solution using the trimesh library.
Also, if you have this same concern as me: vertices generated by marching cubes are very dense, and vertex color is enough for the color resolution.

@torch.no_grad()
def decode_render(
    xm: Union[Transmitter, VectorDecoder],
    latent: torch.Tensor,
    cameras: DifferentiableCameraBatch,
    rendering_mode: str = "stf",
):
    decoded = xm.renderer.render_views(
        AttrDict(cameras=cameras),
        params=(xm.encoder if isinstance(xm, Transmitter) else xm).bottleneck_to_params(
            latent[None]
        ),
        options=AttrDict(rendering_mode=rendering_mode, render_with_direction=False),
    )
    arr = decoded.channels.clamp(0, 255).to(torch.uint8)[0].cpu().numpy()
    return decoded, [Image.fromarray(x) for x in arr]


render_mode = 'stf' # you can change this to 'stf' for mesh rendering
size = 2  # this is the size of the renders; higher values take longer to render.

cameras = create_pan_cameras(size, device)
for i, latent in enumerate(latents):
    decoded, images = decode_render(xm, latent, cameras, rendering_mode=render_mode)
    assert len(decoded['meshes']) == 1
    mesh = decoded['meshes'][0]
    trimesh.exchange.export.export_mesh(trimesh.Trimesh(
        mesh.vertices.cpu().numpy(),
        mesh.faces.cpu().numpy(),
        vertex_colors=mesh.vertex_colors.cpu().numpy()
    ), "barrel-gen.glb")

from shap-e.

Update, If anybody else needs this, see PR #20

And if you want a ply file to display color in blender use this tutorial

from shap-e.

I used meshlab to generate a texture.png from the vertex colors. The export to .obj takes care of the rest.

Ref:
facebookresearch/pifuhd#8 (comment)

from shap-e.

I have a basic alternative solution using the trimesh library. Also, if you have this same concern as me: vertices generated by marching cubes are very dense, and vertex color is enough for the color resolution.

@torch.no_grad()
def decode_render(
    xm: Union[Transmitter, VectorDecoder],
    latent: torch.Tensor,
    cameras: DifferentiableCameraBatch,
    rendering_mode: str = "stf",
):
    decoded = xm.renderer.render_views(
        AttrDict(cameras=cameras),
        params=(xm.encoder if isinstance(xm, Transmitter) else xm).bottleneck_to_params(
            latent[None]
        ),
        options=AttrDict(rendering_mode=rendering_mode, render_with_direction=False),
    )
    arr = decoded.channels.clamp(0, 255).to(torch.uint8)[0].cpu().numpy()
    return decoded, [Image.fromarray(x) for x in arr]


render_mode = 'stf' # you can change this to 'stf' for mesh rendering
size = 2  # this is the size of the renders; higher values take longer to render.

cameras = create_pan_cameras(size, device)
for i, latent in enumerate(latents):
    decoded, images = decode_render(xm, latent, cameras, rendering_mode=render_mode)
    assert len(decoded['meshes']) == 1
    mesh = decoded['meshes'][0]
    trimesh.exchange.export.export_mesh(trimesh.Trimesh(
        mesh.vertices.cpu().numpy(),
        mesh.faces.cpu().numpy(),
        vertex_colors=mesh.vertex_colors.cpu().numpy()
    ), "barrel-gen.glb")

what is 'trimesh.exchange'? it doesn't seem to exist in my library

from shap-e.

couple follow up questions, how could this be implemented using other render modes (the others throw errors)?
how could this be implemented with pytorch3d (texture seems to be implemented differently vs a trimesh)?
can this be added as an example in the project?

from shap-e.

I have a basic alternative solution using the trimesh library. Also, if you have this same concern as me: vertices generated by marching cubes are very dense, and vertex color is enough for the color resolution.

@torch.no_grad()
def decode_render(
    xm: Union[Transmitter, VectorDecoder],
    latent: torch.Tensor,
    cameras: DifferentiableCameraBatch,
    rendering_mode: str = "stf",
):
    decoded = xm.renderer.render_views(
        AttrDict(cameras=cameras),
        params=(xm.encoder if isinstance(xm, Transmitter) else xm).bottleneck_to_params(
            latent[None]
        ),
        options=AttrDict(rendering_mode=rendering_mode, render_with_direction=False),
    )
    arr = decoded.channels.clamp(0, 255).to(torch.uint8)[0].cpu().numpy()
    return decoded, [Image.fromarray(x) for x in arr]


render_mode = 'stf' # you can change this to 'stf' for mesh rendering
size = 2  # this is the size of the renders; higher values take longer to render.

cameras = create_pan_cameras(size, device)
for i, latent in enumerate(latents):
    decoded, images = decode_render(xm, latent, cameras, rendering_mode=render_mode)
    assert len(decoded['meshes']) == 1
    mesh = decoded['meshes'][0]
    trimesh.exchange.export.export_mesh(trimesh.Trimesh(
        mesh.vertices.cpu().numpy(),
        mesh.faces.cpu().numpy(),
        vertex_colors=mesh.vertex_colors.cpu().numpy()
    ), "barrel-gen.glb")

Where do I get Transmitter, VectorDecoder, and DifferentiableCameraBatch?

from shap-e.