No normal gradient (and others I believe) from DSS class about dss HOT 4 CLOSED

Hi, I have same problem as you. And I found another problem https://github.com/yifita/DSS/issues/9

from dss.

I implemented my own dRho computation. I cannot guarantee it is correct, but it hasn't messed anything up for me so far. I replaced lines 276-287 from the rasterize function with this:

        if ctx.needs_input_grad[0]:
            # dI/drho(x) = ( WsMap_ - pixels*h)/(sumRho)
            # Note that this gradient is back-proped to rho, not \overline{rho} (like the paper says)
            # This can be done because computationally the same gradient would occur since the
            # gradient for \overline{rho} at pixels outside the ellipse of rho are all 0.
            WsMap_ = torch.where(isBehind.unsqueeze(-1),
                                 torch.zeros(1, 1, 1, 1, 1, device=WsMap.device, dtype=WsMap.dtype),
                                 WsMap)
            h = (rhoMap_filtered > 0).double() # Easy way to get h function!
            dRho = (WsMap_ - pixels.unsqueeze(3).repeat(1, 1, 1, 5, 1)*h.unsqueeze(-1))/sumRho.unsqueeze(-1)
            dRho = gradPixels.unsqueeze(3)*dRho

            # Invert the mapping so the gradient is back-propped to appropriate BB rho
            # rhoMap = _gather_maps(rho.reshape(batchSize, -1, 1), totalIdxMap, 0.0).squeeze(-1)
            # BATCHES MUST HAVE POINTS IN THE SAME ORDER... OTHERWISE THIS RE-ORDER WILL NOT WORK
            dRho = _guided_scatter_maps(numPoint*bbWidth*bbHeight, dRho,
                                        totalIdxMap, torch.repeat_interleave(boundingBoxes, bbWidth*bbHeight, 1))
            dRho = dRho.reshape(batchSize, -1, bbHeight, bbWidth)
        else:
            dRho = None

Then I removed all "detach()" when computing rho. Hope this helps others!

from dss.

Thank you @bango123 !
I'm curious about your motivation to derive \rho or \bar{\rho}? Do you want to further derive \rho w.r.t. point positions and normals?

from dss.

Hello @yifita! Thank you for releasing this code base and your paper is really high quality btw.

I am doing a research project which is reconstructing from point based geometry and your work seemed like a perfect fit for it. So I put in the time to really understand your work and code implementation.

From your paper, it looks like these are the derivatives wanted (after approximating dh/dn = 0):

And from my understanding of your code, this function (DSS rasterize) is where dI/dw, dI/ \bar{rho}, dI/h are all computed. I noticed dI/ \bar{rho} is never computed, so I tried my hand at implementing it (code above) in an effort to better understand how things work.

However, as you probably already know, it looks like adding \bar{rho} w.r.t to points and normals (i.e. dI/ \bar{rho}) makes very little difference to the gradients. I am guessing that is because your approximated gradient for dI/dh is so much larger in magnitude.

tl;dr wanted to learn about your code/paper, so I tried to fix something. The "fix" appears to make little difference.

from dss.