Use WEBGL_render_shared_exponent for HDR textures about three.js HOT 15 CLOSED

KTX Software can generate E5B9G9R9 from float32 EXRs as well:

ktx create --format E5B9G9R9_UFLOAT_PACK32 --zstd 18 input.exr output.ktx2

ktx create --format B10G11R11_UFLOAT_PACK32 --zstd 18 input.exr output.ktx2

Size on disk looks similar to a compressed EXR, but the memory cost would be lower. We'd probably also need to add support to detect E5B9G9R9 in THREE.KTX2Loader.

from three.js.

Note that browser support is necessary but not sufficient for support on any given device, the graphics drivers also need to support it. Unfortunately overall support is currently 0.07% according to https://web3dsurvey.com/webgl2/extensions/WEBGL_render_shared_exponent. 😞

from three.js.

Just wanted to copy over my comment from #28012 since it might be important for this discussion:

I have tested the change with a modified version of RGBELoader and webgl_loader_texture_hdr. I did not add these changes to this PR since RGBELoader might not be the best spot for using RGB9E5. That's because RGB9E5 is not a color renderable format so we can't convert a RGB9E5 equirectangular environment map to a RGB9E5 cube map. This is required though since we don't support equirectangular textures in the shaders. If we switch to half float in the conversion process, it would not make sense to use RGB9E5 in the first place. The same issue exists with PMREM as well.

from three.js.

Uh, I thought that was the whole point of WEBGL_render_shared_exponent, which says this:

When this extension is enabled:
The RGB9_E5 floating-point internal format becomes color-renderable.

from three.js.

Conformance test.
https://registry.khronos.org/webgl/sdk/tests/conformance2/extensions/webgl-render-shared-exponent.html

from three.js.

More info: see RGB9_E5 https://www.khronos.org/opengl/wiki/Small_Float_Formats

from three.js.

What type of typed array are you use and how do you prepare the data if you want to use RGB9_E5 e.g. with DataTexture?

Is there some sort of reference implementation that we can use similar to the code of DataUtils.toHalfFloat().

I understand the engine has to support the RGB9_E5 format as a constant first so you can configure it. But I'm interested in the actual data so we can easier test the format.

from three.js.

Is there some sort of reference implementation that we can use similar to the code of DataUtils.toHalfFloat().

Maybe from https://registry.khronos.org/OpenGL/extensions/EXT/EXT_texture_shared_exponent.txt:

rgb9e5 float3_to_rgb9e5(const float rgb[3])
{
  rgb9e5 retval;
  float maxrgb;
  int rm, gm, bm;
  float rc, gc, bc;
  int exp_shared;
  double denom;

  rc = ClampRange_for_rgb9e5(rgb[0]);
  gc = ClampRange_for_rgb9e5(rgb[1]);
  bc = ClampRange_for_rgb9e5(rgb[2]);

  maxrgb = MaxOf3(rc, gc, bc);
  exp_shared = Max(-RGB9E5_EXP_BIAS-1, FloorLog2(maxrgb)) + 1 + RGB9E5_EXP_BIAS;
  assert(exp_shared <= RGB9E5_MAX_VALID_BIASED_EXP);
  assert(exp_shared >= 0);
  /* This pow function could be replaced by a table. */
  denom = pow(2, exp_shared - RGB9E5_EXP_BIAS - RGB9E5_MANTISSA_BITS);

  maxm = (int) floor(maxrgb / denom + 0.5);
  if (maxm == MAX_RGB9E5_MANTISSA+1) {
    denom *= 2;
    exp_shared += 1;
    assert(exp_shared <= RGB9E5_MAX_VALID_BIASED_EXP);
  } else {
    assert(maxm <= MAX_RGB9E5_MANTISSA);
  }

  rm = (int) floor(rc / denom + 0.5);
  gm = (int) floor(gc / denom + 0.5);
  bm = (int) floor(bc / denom + 0.5);

  assert(rm <= MAX_RGB9E5_MANTISSA);
  assert(gm <= MAX_RGB9E5_MANTISSA);
  assert(bm <= MAX_RGB9E5_MANTISSA);
  assert(rm >= 0);
  assert(gm >= 0);
  assert(bm >= 0);

  retval.field.r = rm;
  retval.field.g = gm;
  retval.field.b = bm;
  retval.field.biasedexponent = exp_shared;

  return retval;
}

@elalish Does that look right?

from three.js.

I have no idea - figured it would be easier to construct the data in a shader and let the GPU driver handle the format. Might be nice in conjunction with the UltraHDR support @mrdoob is working on, since that'll involve multiplying two SDR textures to get an HDR texture.

from three.js.

Sorry, I should have written my comment more clear. I just wanted to note that I couldn't get the RTT use case to work on any of my devices and yeah the reason is the missing WEBGL_render_shared_exponent support. Hope it improves over time.

from three.js.

Ugh, sorry, I should have checked driver support. I made the error of thinking Safari and Chrome supporting it meant it was already widely available. Well, it'll be nice someday...

from three.js.

Should we revert #28012?

from three.js.

@mrdoob Would you be okay if we add the routines in the addons folder somewhere? As mentioned in the PR RGB9E5 might be interesting for certain projects and the packing/unpacking routines aren't trivial. So it could be good to have them around especially when WEBGL_render_shared_exponent is better supported.

from three.js.

Never mind, I put the code into a Gist. That should be sufficient for now^^.

https://gist.github.com/Mugen87/459649eece614c4e6d57177c138757e1

from three.js.

Closing. We can reconsider producing RGB9E5 textures in loaders when the support for the WEBGL_render_shared_exponent extension gets better.

from three.js.