bkcore / webgl-loader Goto Github PK

When the given decodeParams.decodeScales in the generated JS files (example 
below) is used for downloadModel / downloadMesh the model fails to display 
(can't see anything in the display). However if the default decoding params 
scales it works fine.

Example:
decodeScales: 
[0.205237,0.205237,0.205237,0.000978,0.000978,0.001957,0.001957,0.001957],

Defaults:
decodeScales: [1/8191, 1/8191, 1/8191, 1/1023, 1/1023, 1/1023, 1/1023, 1/1023],


The first three are the furthest off (1680x as large). The others are very 
close (either same or 2x as large). There are some rendering issues (things not 
lining up) and I presume it is because these decoding scales are not right.

Right now, vt only works if there are 2 floats, and doesn't support the 3-float 
version, even if the third component is always 0.0

Builded converter simply by calling it in console.

Then, i run following in trunk folder:
./objcompress pelwis.obj pelwis.utf8

Note, that pelwis.obj already in /trunk

And output:
WARNING: mtllib (?) unsupported at line 4
ERROR: bad texcoord at line 6456

What's this means?

Many real world models have UVs that are outside of <0,1> range. 

Currently such UVs completely break the compression. It would be nice if for 
example compressor normalized incoming "wild" UVs.

There are several things that might make sense to do. I'll throw these together 
in a single bug to keep things simple.

 * Use progress events to incrementally draw partially downloaded meshes
   * bounding boxes
   * vertices without indices as points
   * vertices with indices as triangles
 * Split attributes into multiple streams and refine
   * Pass #1, highly-quantized Delta + ZigZag (as usual)
   * Pass #2+, refinement passes (like morph animation)
 * Do LODs with indices (also helps for rendering)
   * Discrete or continuous LOD? Continuous might be cooler, but has questionable value. Discrete is simpler, and allows for better vertex optimizations
   * Generate new vertices or use existing vertices? New vertices are better quality (especially for textures), but existing vertices are more compact, and possibly simpler.
   * How to encode updates? How to update VBOs?

Should parse MTL files (and understand textures)

Should batch geometry by material

This is to be able to reconstruct original size and world location of the model.

For the moment, this has to go in a different channel (for three.js we dump 
scale and offsets to console in compressor and copy-paste it into our loader), 
it would be nice if UTF8 file was self-contained.

Currently it's actually even less as it seems new vertices can be created in 
vertex optimization stage? (I had it fail on model with original 50K vertices)

This would be very nice, with such great compression it's so tempting to have 
larger models. On decent GPU WebGL can easily handle models with at least 
several hundred thousand triangles, it's pity to be limited by 16 bit indices.

Right now, there is no boundary structure, so it runs in quadratic time.

Lets not make this amateur hour.

Some basic makefiles will make building/testing much simpler and hopefully 
encourage 3rd party development.

Cannot compile with the MSC (Microsoft C Compiler).

The following errors come up:

1. Undeclared identifier 'HUGE_VALF' in optimize.h
2. Undefined function 'strtof()' in mesh.h

The following warnings come up (on warning level 3):

3. 'static_cast' : truncation of constant value utf8.h (x3)
4. 'fopen' : function may be unsafe in mesh.h and objcompress.cc
5. '=' : conversion from 'const float' to 'int', possible loss of data in mesh.h
6. '=' : truncation from 'double' to 'float' in mesh.h  
7. 'argument' : conversion from 'const size_t' to 'float', possible loss of 
data in optimize.h


The following changes should fix these errors/warnings:
1 and 2. Add the following to base.h:
    #ifdef _MSC_VER
    union _MSVC_EVIL_FLOAT_HACK { unsigned __int8 Bytes[4]; float Value; };
    const static union _MSVC_EVIL_FLOAT_HACK _INFINITY_HACK = {{0x00, 0x00, 0x80, 0x7F}};
    #define HUGE_VALF (_INFINITY_HACK.Value)
    // Note: "#define HUGE_VALF (1e25f * 1e25f)" may work as well instead of the above 3 lines
    static inline float strtof(const char * str, char ** endPtr) {
      double d = strtod(str, endPtr);
      return (float)((d > 0 && (d > FLT_MAX || d < FLT_MIN)) ? HUGE_VALF : ((d < 0 && (d < -FLT_MAX || d > -FLT_MIN)) ? -HUGE_VALF : d));
    }
    #endif

3. Switch the static_cast<char> to just a normal char cast.
4. Add "#define _CRT_SECURE_NO_WARNINGS" before any includes in base.h and 
mesh.h
5. Add an int cast:                    ret.decodeOffsets[i] = (int)(maxPosition 
* bounds.mins[i] / scale);
6. Add an "f" after the decimal value: ret.decodeScales[i] = 1.0f / 511;
7. Add a float cast:                   const float valence_boost = 
powf((float)active_tris, -kValenceBoostPower);


Going to warning level 4 introduces 18 more warnings not fixed with the above.

I took the collada duck and converted it to OBJ, then ran objcompress on it to 
get a UTF8 file. I loaded it in three.js and here is the result:
http://jterrace.github.com/webgl-utf8/

The mesh looks good, but the texture is wrong. I thought at first that it might 
just be the texture flipped in the y-axis, but I tried various combinations of 
flipping the texture and couldnt get it to line up correctly.

Is something going wrong here with the encoding? Or perhaps the three.js loader 
is out of date? This is using r50.

What steps will reproduce the problem?

1. Create 3D model with Blender 2.61, export to obj with this options
- Apply Modifiers
- Include Edges
- Include Normals
- Include UVs
- Write Materials
- Objects as OBJ Groups 
2. open-3d-viewer + webgl-loader -> create UTF8, JS, JSON
3. See in open-3D-viewer, but texture don't see

What is the expected output? What do you see instead?
3D model with texture. I see 3D model with white material.

What version of the product are you using? On what operating system?
Last revision of product and Ubuntu 11.10

Normals are often predictable from the positions.

1) Initialize per-vertex normals to zero.
2) For each triangle:
3)   compute the cross product
4)   add the cross product to each triangle vertex normal

This will give you a per-vertex area-weighted normal. Not sure if it needs to 
be normalized at this point (think about what happens when it is interpolated 
per-fragment).

Quick enhancement: instead of initializing normals to 0, initialize to some 
delta from the prediction. Actually, maybe this has to be done as a 
post-normalized fixup...

This technique will be make existing compression slower, so it should be 
optional. However, it can also orthogonal to how you decode the rest of your 
mesh. Obviously, the benefit is that you spend less bits encoding normals.

At the moment, unless I'm missing something, it's not clear how to use the 
output of objcompress to actually do anything. For example, where the JS 
directed to standard out would be pasted in one of the samples in order to 
display the model.

It would be cool if this was documented. It doesn't seem to be trivial.

More examples

Share more code

Interactivity

Benchmark

Add a flag (for example -w) that would signal that a solid white material will 
be used for all missing materials instead of causing and error and the 
compression to fail.

Once again I have a solution for this created already, but it seems as though 
you are not working on this project right now.

third_party/webgl_loader/objanalyze.cc:
cc1plus: warnings being treated as errors
third_party/webgl_loader/objanalyze.cc: In function 'int main(int, char**)':
third_party/webgl_loader/objanalyze.cc:62: error: deprecated conversion from 
string constant to 'char*' [-Wwrite-strings]
third_party/webgl_loader/objanalyze.cc:62: error: deprecated conversion from 
string constant to 'char*' [-Wwrite-strings]
third_party/webgl_loader/objanalyze.cc:62: error: deprecated conversion from 
string constant to 'char*' [-Wwrite-strings]
third_party/webgl_loader/objanalyze.cc:62: error: deprecated conversion from 
string constant to 'char*' [-Wwrite-strings]

Not much of the MTL file is used. Only newmtl, Kd, and map_Kd are supported.

There are numerous other things that can be used in MTL files:

Ka / map_Ka   - ambient color
Ks / map_Ks   - specular color
Ns            - specular color weighting (0 to 1000)
d / Tr        - transparency (0.0 to 1.0)
illum         - illumination model
map_bump / bump
disp
decal
refl

Additionally, the commands that load files (map_*, disp, ...) can take extra 
arguments modifying how those files are used (e.g. -o moves the origin of the 
map).

What steps will reproduce the problem?

If I am loading .obj files either generated from 3ds max exporter or converted 
by the open asset import library i get the error.
WARNING: s ignored in line [linenumber]

as far as i know s stands for the smoothing groups aka vertex normals in the 
.obj file format.

Got this info from here: http://en.wikipedia.org/wiki/Wavefront_.obj_file
"Smooth shading across polygons is enabled by smoothing groups."

Are smoothing groups not recognized in the exporter?

Here is a link to a obj file causing the issue:
http://threever.org/demos/problem/waranHigh.obj

When i trying to use UTF8Loader (from three.js lib) with utf8 models converted 
by webgl-loader, always getting this error:

ArrayBufferView size is not a small enough positive integer
THREE.UTF8Loader.decompressMesh   Three.js:645
m   Three.js:646
THREE.UTF8Loader.createModel   Three.js:648
c.onreadystatechange   Three.js:644


------------------
using r45 of Three.js and r61 (last) of webgl-loader.

What is this?

Hi, I'm working in a project use Three.js with your webgl-loader (utf8).

Your extension is super amazing with about 20 time smaller, i never can imagine 
that, but when i almost done my project so i realize this problem: 

Utf8 model got 'deviance' at UV seam? 

Please look at this question i'm asking in Stackoverflow: 

    http://stackoverflow.com/questions/22129257/utf8-model-got-deviance-at-uv-seam

Can you see and fix this problem please, i'm waiting for this to publish my 
project?  

Thank you very very much.

Hi, Won, it's me again :)

I encountered another problem, - when we tried to load more than 1 model, we 
noticed, that model coordinates not exported, - all models positioned in center.

You can see this in attachment screen. We're waiting for your help, friend.

Was thinking about the quantization step encoding floats as 10-14 bits. Was 
looking at lossless float compression and came across:
http://www.cs.unc.edu/~isenburg/lcpfpv/

Might be interesting to try implementing something like this for encoding the 
float arrays to avoid loss in precision. Might be able to get a nice savings at 
a small cost in decoding.

Currently, indices of the form: a//b (no texcoord) fail.

Reported internally.

A flag should be added (for example -m) that would cause the output to be 
minified:

No whitespace outside of strings.
No superfluous commas (commas before } and ]). These could be removed anyways.

I am working on a solution to this.

The demos have a problem. Shader attributes aren't properly handled: 
specifically, different WebGL implementations do different things for 
un-referenced attributes. I should either add textures, or use texture 
coordinates.

Storing XYZ for normals is expensive, considering they are normalized. How 
about a more compact representation? For example, .MD2 used an <8-bit code to 
select per-vertex normals out of a palette.

The downside of a palette is that you have to send the palette. What about 
something that used [http://en.wikipedia.org/wiki/Peirce_quincuncial_projection 
a simple sphere projection]?

Triangle strips require fewer indices due to implicit connectivity. This could 
halve the number of indices required. One downside is that you have to deal 
with restarts and/or degenerate triangles.

One benefit from the implicit connectivity is that it gives you a convenient 
way to perform parallelogram prediction, which should be better than delta.

I tried MinGW on Windows (4.5.2) and default GCC version that came on Ubuntu 
11.10 (4.6.1). 

Both complained about "strpbrk" and "strncmp" not being declared. 

For example here:

base.h:77:39: error: 'strpbrk' was not declared in this scope
mesh.h:425:46: error: 'strncmp' was not declared in this scope

MinGW additionally complains about unknown "z" format in printf.

For example here:

mesh.h:513:65: warning: unknown conversion type character 'z' in format
objcompress.cc:120:20: warning: unknown conversion type character 'z' in format

What is a preferred way to build objcompress executable on Windows / Linux?

Write some code to compute ACMR/ATVR for triangle meshes. Think about ways to 
simulate vertex transforms for non FIFO-based caches, like the block-based 
caches for unified shader architecture, or whatever tile-based renderers do.

Compiling with MinGW (GCC for Windows) still uses the Microsoft C libraries 
which use %Iu instead of %zu for size_t format specification.

Recommendation is to add the following to base.h:

#ifdef _WIN32 // defined in MSVC and MinGW
#define SIZET_FORMAT "%Iu"
#else
#define SIZET_FORMAT "%zu"
#endif

Then replace all instances of %zu with "SIZET_FORMAT" (keeping quotes, except 
in one case the first quote needs to be removed because the string starts with 
%zu).

This format maybe could be extended to efficiently compress animations done as 
morphing between vertex positions (vertex / face numbers and order must be the 
same in every frame).

Base frame would be exactly like it already is, then new frames would need just 
deltas for each vertex position (assumption is deltas would be usually very 
small, each new frame is similar to previous one).

Not sure if vertex optimization phase wouldn't mess with vertex order / create 
different numbers of vertices in each frame?

When the attached OBJ file is run with:

objcompress 'Brain.obj' 'Brain.utf8'

it fails with the following message:

mesh.h:940 CHECK failed: Uint16ToUtf8(za, utf8)

Since I have modified my source code, the line number is incorrect, however it 
is the CHECK() in CompressQuantizedAttribsToUtf8(). I have only modified MTL 
parser and JSON writing. The OBJ parser, UTF8 stuff, or binary writing 
functions have been modified at all.

The attached file (and many others like it) cannot be loaded, the output to 
stderr (on Windows 7 x64 compiled with MinGW) is:


terminate called after throwing an instance of 'std::out_of_range'
  what():  vector::_M_range_check

This application has requested the Runtime to terminate it in an unusual way.
Please contact the application's support team for more information.


The file is created with IMOD. If exporting from IMOD without normals then the 
file converts fine (although I am having problems getting it to render with a 
material).

Right now, the arguments to objcompress can't be path + filename, just 
filenames. This is not ideal, especially when running from a process.

What steps will reproduce the problem?

Loading in a large mesh that was converted from assimp (open asset import lib) 
to .obj produces the following issue:

ERROR: bad second index at line [...]

Looking in the obj file at the given line revealed nothing special to me:

#0 vertex normals

# Mesh 'defaultobject' with 1388088 faces
g defaultobject
usemtl DefaultMaterial
f  1/1/ 2/2/ 3/3/ <<<<<<<<< exporter : ERROR: bad second index
f  4/4/ 5/5/ 6/6/
f  7/7/ 8/8/ 9/9/
f  10/10/ 11/11/ 12/12/

Model file:
http://threever.org/demos/problem/dScanASSIMP.obj

mtl lib file:
http://threever.org/demos/problem/dScan.obj.mtl

(Possible something went wrong with the converting in assimp...)

Here is the stack trace:

{{{
(gdb) run duck.obj duck.utf8
Starting program: /home/jterrace/webgl-loader/bin/objcompress duck.obj duck.utf8

Program received signal SIGSEGV, Segmentation fault.
0x00000000004040e8 in DrawBatch::AddTriangle (this=0x62a3e8, group_line=0, 
indices=0x7fffffffd940) at ../src/mesh.h:325
325         if (flat_index > group.max_index) {
(gdb) bt
#0  0x00000000004040e8 in DrawBatch::AddTriangle (this=0x62a3e8, group_line=0, 
indices=0x7fffffffd940) at ../src/mesh.h:325
#1  0x00000000004052b3 in WavefrontObjFile::ParseFace (this=0x7fffffffdba0, 
line=0x7fffffffda08 "", line_num=8549) at ../src/mesh.h:641
#2  0x0000000000404e5d in WavefrontObjFile::ParseLine (this=0x7fffffffdba0, 
line=0x7fffffffd9e0 "f 235/3272/610 271/3255/825 348/3230/876", line_num=8549)
    at ../src/mesh.h:536
#3  0x0000000000404d58 in WavefrontObjFile::ParseFile (this=0x7fffffffdba0, 
fp=0x627010) at ../src/mesh.h:526
#4  0x0000000000404aa7 in WavefrontObjFile::WavefrontObjFile 
(this=0x7fffffffdba0, fp=0x627010) at ../src/mesh.h:479
#5  0x000000000040242e in main (argc=3, argv=0x7fffffffe1e8) at 
../src/objcompress.cc:30
}}}

Currently, not all formats are correctly handled, such as:

v x y z w  # projective points

v x y z r g b # per-vertex color

vt u v w  # projective texcoords

There should be a way for meshes to describe its vertex format (and possibly be 
overridden).