Relighting problems about neroic HOT 2 CLOSED

Hi Menpinland,

1. Good question! Thanks for pointing this out. The main reason why our model does not produce quite good relighting in this case is that our model are based on low level spherical-harmonics, which is designed to render images with very ambient lightings. Since you are using a lighting map with a very strong source (the sun), it will drasically affect the approximation from the SHs, and thus causing the artifact. An easy solution to that is to replace the SH-based rendering to other methods, such as SG-based rendering or just brute-force, and there're many resources you can look into. In the meantime, you can also try with some more ambient lighting, such as a cloudy outdoor scene, or an indoor scene with ambient lights.
2. I'm not quite sure what's the question here. The self.env_lights is used during the rendering at here and thus trained with the back-propagated gradients. Let me know if this answers your question.

from neroic.

Hi Menpinland,

1. Good question! Thanks for pointing this out. The main reason why our model does not produce quite good relighting in this case is that our model are based on low level spherical-harmonics, which is designed to render images with very ambient lightings. Since you are using a lighting map with a very strong source (the sun), it will drasically affect the approximation from the SHs, and thus causing the artifact. An easy solution to that is to replace the SH-based rendering to other methods, such as SG-based rendering or just brute-force, and there're many resources you can look into. In the meantime, you can also try with some more ambient lighting, such as a cloudy outdoor scene, or an indoor scene with ambient lights.
2. I'm not quite sure what's the question here. The self.env_lights is used during the rendering at here and thus trained with the back-propagated gradients. Let me know if this answers your question.

Thank you for your answer. I will try to get familiar with more lighting models and make more attempts. For the 2rd problem, I have misunderstood some details in the paper and caused confusion.

from neroic.