Low level of details for synthetic dragon about iron HOT 6 OPEN

Hi -- can you share some of your rendered images?

Note that we do gamma-correct our mitsuba-rendered images (also w/ proper global scaling to reasonable brightness) and save as uint8 png images. Then we will use these png images (no inverse gamma correction) in the first stage; in the second stage, we will first inverse-gamma-correct the png images, as rendering equation is linear.

from iron.

Hello!
Without gamma-correction mitsuba-rendered images looks like under-exposed:

For the first stage I used this code for reading *.exr files, which performs gamma-correction to reasonable brightness:

Is it flag --inv_gamma_gt supposed to be set during the second stage to perform inverse gamma-correction of the input images? I'm asking, because train_scene.sh script doesn't set this flag for the second stage.

from iron.

Hi -- the exr images need to be scaled (same scaling factor applied to all your training and test images) and then gamma-corrected to have well-exposed uint8 png images. I can share some example script to achieve this exr to png conversion later.

And yes, --inv_gamma_gt would be needed in the second stage for the synthetic data. (the setting in train_scene.sh was meant for real-world images; somehow I found that without this --inv_gamma_gt, the method recovers better specularity for real-world captures sometimes).

from iron.

Hello!
Thank you for the reply!
Could you suggest any reasonable scaling factor to apply for all exr images before gamma-correction (with grade 1/2.2)?

from iron.

Let's say you put all your training and test images into a tensor of shape [N, H, W, 3]; then a good heuristic might be to scale the 98-percentile (might worth trying other percentiles) of the flattened tensor to be 1; then you clip the scaled tensor to the range (0,1), and perform gamma correction, followed by saving as uint8 png.

from iron.

Hi!
I experimented a bit with the different percentiles for normalization and gamma-correction setups for the second stage. The best result I got is with 95-percentile and disabled --gamma_pred and enabled --inv_gamma_gt flags. Unfortunately, this result is still doesn't contain small details and sharp edges like that presented in the paper.


Do you have any suggestion how to tune configs/setups to get the shape more accurate?
Thanks!

from iron.