Extend affinder to 3D about affinder HOT 2 OPEN

Hello @jni,
First I'd like to thank you for your kind and detailed reply.
So far I just wanted to give you a feedback on the fact that I managed to use affinder with my 3D images in the 3D view mode (MIP). I first added landmark "points" in the dorsal plane, then a few others in transverse and median plane.

This gave a first rough alignment. I will check on the details of your answer to see how I could contribute to this plugin.

Best regards,
Marine

from affinder.

Hello @marbre2!

Thanks for opening this issue!

So, in theory, it should actually already work in 3D: all of the alignment and callback code makes no assumptions about the dimensionality of the underlying images. The three issues are that:

• adding correspondence points in 3D is tricky / doesn't work at all; we could potentially do it with a similar method to the 3D filament annotator: https://twitter.com/anna_medyukhina/status/1578114418805800976 ie, click once to create a "ray", rotate the view, click again to select a point within the ray. I'd be super excited to help with that.
• napari doesn't handle non-orthogonal transformations in 3D when displaying 2D slices. That is, if you e.g. rotate a volume 45º along the xz plane, when you look at xy napari will show orthogonal planes to z. Therefore, if you are using the 2D view (which is the only view where you can currently add points), the images won't be aligned correctly, which is kinda sorta the whole point with affinder. 😂
• The other whole point with affinder 😂 is to set the view to center on the predicted location of your next click, once you have an initial alignment. I think that probably doesn't work in 3D view; in 2D view, which as we said above is flawed, it could be made to work, but we should also use dims.set_point in addition to setting the viewbox.

So, in summary, those are the challenges, but the alignment stuff should basically work; contributions would be super super welcome and I would be happy to help, give you a tour of the code if anything's unclear (😬) (omg just saw that the main function doesn't even have a docstring yet 😂 😭), etc!

PS: You might also be interested in @thanushipeiris's in-progress PR to allow mixed-dimension registration, #58.

from affinder.