MOC overlap with a MRM: calculation of proba in the intersection about mocpy HOT 14 CLOSED

@ManonMarchand Yeah. After you have read in the skymap with read_sky_map from ligo.skymap.io import read_sky_map, as skymap = read_sky_map(filename, moc=True, distances=True), for example.

from mocpy.

@ManonMarchand for the Rust part, as a starting point, see the new branch mom_sum.

from mocpy.

Hi @mcoughlin, thank you for your message.

So far, there is no way to do this.
But, I think it can be implemented quite easily.

If I understand correctly, you want to perform the intersection between a MOC and a MultiResolution HEALPix Map (MRM)
and sum the values in the resulting (sub-)MRM.
If a MOC cell is included in a larger MRM cell, I assume that the probability associated with the MRM cell must be multiply by the ratio of the area of the MOC cell over the area of the MRM cell, right?

What about a method: MOC.sum_of_multi_order_map_values(path_to_the_mom_fits_file)?

from mocpy.

@fxpineau exactly. I think it should probably take in a table assuming you already loaded the fits file into memory as you will likely do this many times for different fields, but that is a minor nitpick. Thanks!

from mocpy.

additional TODO:

document MOC.from_multiordermap_fits_file and replace the call to astropy_healpix in the bayestar.py example

from mocpy.

@mcoughlin what kind of table do you have? Is it an ~astropy.table.Table?

from mocpy.

@mcoughlin, @ManonMarchand,

I just added the functionality in MOC Lib Rust (see this commit) and in moc-cli (see this commit) but it is not released yet (only pushed in the main branch).

We will try to integrate the functionality in MOCPy next week (we are in holidays for the rest of the week).

from mocpy.

Thanks @fxpineau. We look forward to the integration in mocpy!

from mocpy.

Hi,

We added two methods (not yet merged but available in PR #127):

• MOC.sum_in_multiordermap that takes a table and a column name in arguments and returns the sum of the column given in the intersection between the MOM and the MOC
• MOC.probability_in_multiordermap that will convert a PROBDENSITY column into a probability in the intersection directly.

Is it what you needed?

from mocpy.

Thanks @ManonMarchand!

I would guess you have confirmed that you get the same answer with your function as when you flatten both the skymap and the moc and do the sum that way?

One idea (for the sake of potential speed up) is for a function that takes many MOCs and a skymap and returns an array of probabilities (one for each MOC). I suspect most folks will need to be looping over many fields so giving them a function where you do that internally is likely to be very useful.

from mocpy.

Thank you for your feedback @mcoughlin.

I would guess you have confirmed that you get the same answer with your function as when you flatten both the skymap and the moc and do the sum that way?

We basically checked that the value returned by the function applied to a MOC created from a multi-order map + a probability threshold, and taking as parameter the same input multi-order map, fit with the probability threshold.
For additional tests requiring operations on multi-order map (such a flattening), I think they should be integrated to the multi-order map library (we would like not to add a dependency to the multi-order map library in MOCPy).

[...] a function that takes many MOCs and a skymap and returns an array of probabilities (one for each MOC). [...]

From my point-of-view, this should be a method of the multi-order map object, thus in the multi-order map library (I also wonder if the PROBDENSITY to Proba conversion should not be in the multi-order map library instead of the specific probability_in_multiordermap MOCPy method): the conversion from PROBDENSITY to proba could be performed only once and then internally call MOC.sum_in_multiordermap for each MOC (I tend to think that iterating n times over the mutli-order map will be faster than iterating only once and making for each cell a call on each MOC (depending on the MOM + MOCs size with respect to the CPU cache size), and it is quite easy to multithread in case of performance issues).

Let first test and see if performances have to be improved for multiple MOCs.
Don't hesitate to open a new issue if you were to find that the performances don't allow you to do your calculations.

from mocpy.

@fxpineau @ManonMarchand Awesome. I would be happy to test this ASAP, especially if you could get a version up on pypi.

from mocpy.

Thanks for the pypi release. Is the same planned for conda?

from mocpy.

Yep, done: https://anaconda.org/conda-forge/mocpy

from mocpy.