cds-astro / mocpy Goto Github PK

Python library to easily create and manipulate MOCs (Multi-Order Coverage maps)

Home Page: https://cds-astro.github.io/mocpy/

License: BSD 3-Clause "New" or "Revised" License

Python 76.29% Rust 23.71%

astronomy astronomy-astrophysics tesselation healpix moc astrophysics astropy-affiliated gravitational-waves open-source python

mocpy's People

Stargazers

Watchers

mocpy's Issues

Command-line script for MOC generation / filtering / etc

It would be nice to have some command-line scripts for some common operations as:

MOC generation from a list of positions
catalogue filtering by a MOC
MOC operations (union/intersection)

moving mocpy docs causing sphinx failures

mocpy docs has been moved from https://mocpy.readthedocs.io/en/latest/ to https://cds-astro.github.io/mocpy/ thus causing docs build failures for astroquery.

I'm fixing it for master, but you might want to double check on github where else was the old url used.

Angle object for inputs in MOC.add_position?

In https://github.com/cds-astro/mocpy/pull/15/files#diff-a40381ce2439dfbb7713590bad6e5b75R53 I noticed that in MOC.add_position the docstring says that Angle objects are the input for ra and dec, but then the current implementation (both before and after this change) assumes that it's floats or float arrays in deg.

Suggest to change to quantity inputs like in the astropy_healpix API.
Requires adapting callers, and maybe even increate test coverage a bit before changing.

mocpy does not work in python 3.4

Hello,

I am interested in using mocpy, but rather would like to work in python 3.4 rather than in 2.7. I have a MOC file that I am trying to read. In python 2.7 it works fine:
Python 2.7.11 (default, Dec 5 2015, 23:51:51)
Type "copyright", "credits" or "license" for more information.

IPython 4.0.1 -- An enhanced Interactive Python.
? -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help -> Python's own help system.
object? -> Details about 'object', use 'object??' for extra details.

In [1]: from mocpy import MOC

In [2]: mc = MOC.from_file('/Users/haussel/Downloads/Herschel-Stripe-82_MOC.fits’)

In python 3.4 I get the following error:

Python 3.4.4 (default, Dec 21 2015, 04:54:59)
Type "copyright", "credits" or "license" for more information.

In [1]: from mocpy import MOC

In [2]: mc = MOC.from_file('/Users/haussel/Downloads/Herschel-Stripe-82_MOC.fits')

TypeError Traceback (most recent call last)
in ()
----> 1 mc = MOC.from_file('/Users/haussel/Downloads/Herschel-Stripe-82_MOC.fits')

/Users/haussel/Library/Python/3.4/lib/python/site-packages/mocpy/moc.py in from_file(cls, local_path)
204 @classmethod
205 def from_file(cls, local_path):
--> 206 return MOC_io.read_local(local_path)
207
208

/Users/haussel/Library/Python/3.4/lib/python/site-packages/mocpy/moc.py in read_local(path)
492 Read a MOC on the local file system
493 """
--> 494 return MOC_io.__parse(path)
495
496

/Users/haussel/Library/Python/3.4/lib/python/site-packages/mocpy/moc.py in __parse(path)
508
509
--> 510 return MOC.from_uniq_interval_set(interval_set)
511
512

/Users/haussel/Library/Python/3.4/lib/python/site-packages/mocpy/moc.py in from_uniq_interval_set(cls, uniq_is)
336
337 if order != last_order:
--> 338 r = r.union(rtmp)
339 rtmp.clear()
340 last_order = order

/Users/haussel/Library/Python/3.4/lib/python/site-packages/mocpy/interval_set.py in union(self, another_is)
106 """
107 interval_set = IntervalSet()
--> 108 interval_set._intervals = IntervalSet.merge_intervals(self.intervals + another_is.intervals)
109
110 return interval_set

/Users/haussel/Library/Python/3.4/lib/python/site-packages/mocpy/interval_set.py in merge_intervals(intervals)
47 ret = []
48 start = stop = None
---> 49 for itv in sorted(intervals):
50 if start is None:
51 start, stop = itv

TypeError: unorderable types: float() < NoneType()

STMOC from Polygon

There is an existing method to create MOC from a polygon (MOC.from_polygon). We should have a similar method to do the same with ST-MOC. This would be useful when creating STMOC from sets of footprints, where we only wish to describe the boundaries in both space and time.

Getting the list of pixels

I noticed that there is no access method or property to get the underlaying list of pixels, is there? Any reason not to have one? I'm working on a project where I ended up needing this.

Or in other words, why is _uniq_format() not a "public" method? Maybe I'm just missing the proper way to access this information, sorry if that's the case.

STMOC `from_fits` not closing FITS

As title, the FITS is left open after calling STMOC.from_fits().

Add support to read MOC from JSON serialization

JSON serialization is described in section 3.1 of standard document http://ivoa.net/documents/MOC/20140602/REC-MOC-1.0-20140602.pdf

Improve union method for a list of MOC

It could be nice to improve the union method for a list of MOC - MOC.union(list).

Improve (a lot) the expand method

Using OwnedRangeMOC in cdshealpix-rust

perimeter extraction

I suggest to build a new function to extract the perimeter of a MOC as in the Aladin Desktop -
export -ajs moc.fits. We can use it in several applications for specific plots.

best,
Giuseppe

Generate a MOC from a polygon

See for inspiration the same feature in Aladin Desktop.

Use uniq methods from astropy-healpix

Use uniq methods from astropy-healpix and remove the one in utils.py as it has been implemented in
astropy/astropy-healpix#105

MOC.from_polygon(): pyo3_runtime.PanicException: called `Option::unwrap()` on a `None` value

I seem to be having a wrap issue making polygons for areas of the sky around a Dec of +00.

For example I try to make the following MOC using sky coords shown in this plot (the skycoords make up the blue points around the border):

it errors with the output below.

I have other fields like this one and they all run fine apart from the ones at +00 dec, failures:

0918+00
1147+00
2004+00

Python 3.7.4
MOCPy              0.8.3
cdshealpix         0.5.2

 moc = MOC.from_polygon_skycoord(coords, max_depth=9)

thread '<unnamed>' panicked at 'called `Option::unwrap()` on a `None` value', /root/.cargo/git/checkouts/cds-healpix-rust-1e389e7afcf43397/36c794c/src/special_points_finder.rs:209:13
stack backtrace:
   0:     0x7f6ba303d8b5 - backtrace::backtrace::libunwind::trace::h934f9000aebd3cdb
                               at /cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.46/src/backtrace/libunwind.rs:86
   1:     0x7f6ba303d8b5 - backtrace::backtrace::trace_unsynchronized::hcd6e39da2d76d8d6
                               at /cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.46/src/backtrace/mod.rs:66
   2:     0x7f6ba303d8b5 - std::sys_common::backtrace::_print_fmt::ha0ad3a34f556be49
                               at src/libstd/sys_common/backtrace.rs:78
   3:     0x7f6ba303d8b5 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::h8b081861d17846db
                               at src/libstd/sys_common/backtrace.rs:59
   4:     0x7f6ba305baec - core::fmt::write::hdf236390fbd68d3d
                               at src/libcore/fmt/mod.rs:1076
   5:     0x7f6ba303bcf3 - std::io::Write::write_fmt::h5bb44e52de2a261a
                               at src/libstd/io/mod.rs:1537
   6:     0x7f6ba303fe70 - std::sys_common::backtrace::_print::ha4c1f300ce7771d8
                               at src/libstd/sys_common/backtrace.rs:62
   7:     0x7f6ba303fe70 - std::sys_common::backtrace::print::h91852b42a347b2e6
                               at src/libstd/sys_common/backtrace.rs:49
   8:     0x7f6ba303fe70 - std::panicking::default_hook::{{closure}}::h2fe76e4c26a76fb6
                               at src/libstd/panicking.rs:198
   9:     0x7f6ba303fbbc - std::panicking::default_hook::h2c17754d359ee45d
                               at src/libstd/panicking.rs:218
  10:     0x7f6ba30404a7 - std::panicking::rust_panic_with_hook::h57453d1f9e90d130
                               at src/libstd/panicking.rs:486
  11:     0x7f6ba30400ab - rust_begin_unwind
                               at src/libstd/panicking.rs:388
  12:     0x7f6ba305aef1 - core::panicking::panic_fmt::hd101a87121fa411f
                               at src/libcore/panicking.rs:101
  13:     0x7f6ba305ae3d - core::panicking::panic::h02171c407fa1462f
                               at src/libcore/panicking.rs:56
  14:     0x7f6ba301d403 - cdshealpix::special_points_finder::arc_special_point_in_eqr::hc448bce441fb7041
  15:     0x7f6ba301c8ed - cdshealpix::special_points_finder::arc_special_points::h9a311b3a3832ca4a
  16:     0x7f6ba3015609 - cdshealpix::nested::Layer::polygon_coverage::hae121018a24f79a7
  17:     0x7f6ba2ff4f4a - cdshealpix::cdshealpix::polygon_search::h794c282a6e21fcb1
  18:     0x7f6ba2febd87 - cdshealpix::cdshealpix::__pyo3_get_function_polygon_search::__wrap::{{closure}}::h1c96c6609f1202f1
  19:     0x7f6ba2ff4924 - cdshealpix::cdshealpix::__pyo3_get_function_polygon_search::__wrap::h1e16695d5bbf4113
  20:     0x558a90875744 - _PyMethodDef_RawFastCallKeywords
                               at /tmp/build/80754af9/python_1565725737370/work/Objects/call.c:694
  21:     0x558a90875861 - _PyCFunction_FastCallKeywords
                               at /tmp/build/80754af9/python_1565725737370/work/Objects/call.c:734
  22:     0x558a908e12bd - call_function
                               at /tmp/build/80754af9/python_1565725737370/work/Python/ceval.c:4568
  23:     0x558a908e12bd - _PyEval_EvalFrameDefault
                               at /tmp/build/80754af9/python_1565725737370/work/Python/ceval.c:3093
  24:     0x558a90825539 - _PyEval_EvalCodeWithName
                               at /tmp/build/80754af9/python_1565725737370/work/Python/ceval.c:3930
  25:     0x558a90874ef5 - _PyFunction_FastCallKeywords
                               at /tmp/build/80754af9/python_1565725737370/work/Objects/call.c:433
  26:     0x558a908e0f29 - call_function
                               at /tmp/build/80754af9/python_1565725737370/work/Python/ceval.c:4616
  27:     0x558a908e0f29 - _PyEval_EvalFrameDefault
                               at /tmp/build/80754af9/python_1565725737370/work/Python/ceval.c:3093
  28:     0x558a90825539 - _PyEval_EvalCodeWithName
                               at /tmp/build/80754af9/python_1565725737370/work/Python/ceval.c:3930
  29:     0x558a90874f57 - _PyFunction_FastCallKeywords
                               at /tmp/build/80754af9/python_1565725737370/work/Objects/call.c:433
  30:     0x558a908dd8cc - call_function
                               at /tmp/build/80754af9/python_1565725737370/work/Python/ceval.c:4616
  31:     0x558a908dd8cc - _PyEval_EvalFrameDefault
                               at /tmp/build/80754af9/python_1565725737370/work/Python/ceval.c:3139
  32:     0x558a90825539 - _PyEval_EvalCodeWithName
                               at /tmp/build/80754af9/python_1565725737370/work/Python/ceval.c:3930
  33:     0x558a90874f57 - _PyFunction_FastCallKeywords
                               at /tmp/build/80754af9/python_1565725737370/work/Objects/call.c:433
  34:     0x558a908dd8cc - call_function
                               at /tmp/build/80754af9/python_1565725737370/work/Python/ceval.c:4616
  35:     0x558a908dd8cc - _PyEval_EvalFrameDefault
                               at /tmp/build/80754af9/python_1565725737370/work/Python/ceval.c:3139
  36:     0x558a90874ccb - function_code_fastcall
                               at /tmp/build/80754af9/python_1565725737370/work/Objects/call.c:283
  37:     0x558a90874ccb - _PyFunction_FastCallKeywords
                               at /tmp/build/80754af9/python_1565725737370/work/Objects/call.c:408
  38:     0x558a908dc806 - call_function
                               at /tmp/build/80754af9/python_1565725737370/work/Python/ceval.c:4616
  39:     0x558a908dc806 - _PyEval_EvalFrameDefault
                               at /tmp/build/80754af9/python_1565725737370/work/Python/ceval.c:3124
  40:     0x558a90825539 - _PyEval_EvalCodeWithName
                               at /tmp/build/80754af9/python_1565725737370/work/Python/ceval.c:3930
  41:     0x558a90826424 - PyEval_EvalCodeEx
                               at /tmp/build/80754af9/python_1565725737370/work/Python/ceval.c:3959
  42:     0x558a9082644c - PyEval_EvalCode
                               at /tmp/build/80754af9/python_1565725737370/work/Python/ceval.c:524
  43:     0x558a9093bb74 - run_mod
                               at /tmp/build/80754af9/python_1565725737370/work/Python/pythonrun.c:1035
  44:     0x558a90945eb1 - PyRun_FileExFlags
                               at /tmp/build/80754af9/python_1565725737370/work/Python/pythonrun.c:988
  45:     0x558a909460a3 - PyRun_SimpleFileExFlags
                               at /tmp/build/80754af9/python_1565725737370/work/Python/pythonrun.c:429
  46:     0x558a90947195 - pymain_run_file
                               at /tmp/build/80754af9/python_1565725737370/work/Modules/main.c:433
  47:     0x558a90947195 - pymain_run_filename
                               at /tmp/build/80754af9/python_1565725737370/work/Modules/main.c:1612
  48:     0x558a90947195 - pymain_run_python
                               at /tmp/build/80754af9/python_1565725737370/work/Modules/main.c:2873
  49:     0x558a90947195 - pymain_main
                               at /tmp/build/80754af9/python_1565725737370/work/Modules/main.c:3413
  50:     0x558a909472bc - _Py_UnixMain
                               at /tmp/build/80754af9/python_1565725737370/work/Modules/main.c:3448
  51:     0x7f6bee483545 - __libc_start_main
  52:     0x558a908ec062 - <unknown>
                               at ../sysdeps/x86_64/elf/start.S:103
Traceback (most recent call last):
  File "gen_mocs.py", line 69, in <module>
    themoc = gen_moc(f)
  File "gen_mocs.py", line 51, in gen_moc
    moc = MOC.from_polygon_skycoord(coords, max_depth=9)
  File "/suphys/aste7152/.conda/envs/astropy3/lib/python3.7/site-packages/mocpy/moc/moc.py", line 688, in from_polygon_skycoord
    return MOC.from_polygon(lon=skycoord.icrs.ra, lat=skycoord.icrs.dec, max_depth=max_depth)
  File "/suphys/aste7152/.conda/envs/astropy3/lib/python3.7/site-packages/mocpy/moc/moc.py", line 714, in from_polygon
    pix, depth, fully_covered_flags = cdshealpix.polygon_search(lon, lat, max_depth)
  File "/suphys/aste7152/.conda/envs/astropy3/lib/python3.7/site-packages/cdshealpix/nested/healpix.py", line 582, in polygon_search
    ipix, depth, full = cdshealpix.polygon_search(depth, lon, lat, flat)
pyo3_runtime.PanicException: called `Option::unwrap()` on a `None` value

MOC.from_polygon_skycoord aborts

This is with mocpy 0.8.1, cdshealpix 0.5.0, Python 3.8.1, Numpy 1.18.1.

$ RUST_BACKTRACE=full python
Python 3.8.1 (default, Jan  7 2020, 01:35:47) 
[Clang 11.0.0 (clang-1100.0.33.12)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from mocpy import MOC
>>> from astropy.coordinates import SkyCoord
>>> from astropy import units as u
>>> coords = SkyCoord([(353.8156714, -56.33202193), (6.1843286, -56.33202193), (5.27558041, -49.49378172), (354.72441959, -49.49378172)], unit=u.deg)
>>> MOC.from_polygon_skycoord(coords)
thread '<unnamed>' panicked at 'Wrong hash value: too large.', /Users/travis/.cargo/git/checkouts/cds-healpix-rust-1e389e7afcf43397/8812d1c/src/nested/mod.rs:1920:37
stack backtrace:
   0:        0x10e212a0f - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::had9354ae1d34d490
   1:        0x10e22a74e - core::fmt::write::h481ad6cb6e3feecb
   2:        0x10e210deb - std::io::Write::write_fmt::h6b4d75eb32163e17
   3:        0x10e21474a - std::panicking::default_hook::{{closure}}::h856f143d631a17ca
   4:        0x10e214472 - std::panicking::default_hook::h1e63b9d2abe1ab64
   5:        0x10e214df8 - std::panicking::rust_panic_with_hook::hebcb37a5ea7b3619
   6:        0x10e1f144d - std::panicking::begin_panic::h8d7fdb0ce724bde1
   7:        0x10e1edbed - cdshealpix::nested::Layer::polygon_coverage::ha6a4e48f572306a8
   8:        0x10e1d3574 - cdshealpix::cdshealpix::__pyo3_get_function_polygon_search::__wrap::h79e56dbab67884d1
   9:        0x101055b5a - cfunction_call_varargs
  10:        0x101055649 - _PyObject_MakeTpCall
  11:        0x1010f44e3 - call_function
  12:        0x1010ed714 - _PyEval_EvalFrameDefault
  13:        0x1010f4edd - _PyEval_EvalCodeWithName
  14:        0x101055fe9 - _PyFunction_Vectorcall
  15:        0x1010f4319 - call_function
  16:        0x1010ed714 - _PyEval_EvalFrameDefault
  17:        0x1010f4edd - _PyEval_EvalCodeWithName
  18:        0x101055fe9 - _PyFunction_Vectorcall
  19:        0x101057f31 - method_vectorcall
  20:        0x1010f4319 - call_function
  21:        0x1010ed838 - _PyEval_EvalFrameDefault
  22:        0x1010f4edd - _PyEval_EvalCodeWithName
  23:        0x101055fe9 - _PyFunction_Vectorcall
  24:        0x101057f31 - method_vectorcall
  25:        0x1010f4319 - call_function
  26:        0x1010ed714 - _PyEval_EvalFrameDefault
  27:        0x1010f4edd - _PyEval_EvalCodeWithName
  28:        0x1010eb7b4 - PyEval_EvalCode
  29:        0x1011235a1 - run_eval_code_obj
  30:        0x1011229ef - run_mod
  31:        0x101121548 - PyRun_InteractiveOneObjectEx
  32:        0x101120d6f - PyRun_InteractiveLoopFlags
  33:        0x101120cc0 - PyRun_AnyFileExFlags
  34:        0x1011389af - Py_RunMain
  35:        0x101138b9c - pymain_main
  36:        0x101138bf1 - Py_BytesMain
fatal runtime error: failed to initiate panic, error 5
Abort trap: 6

inMoc and nearMoc methods from STILTS

I noted interesting methods from the STILTS VO tool that can be useful in mocpy. They are InMOC (Indicates whether a given sky position falls strictly within a given MOC) and NearMOC (Indicates whether a given sky position either falls within, or is within a certain distance of the edge of a given MOC). The first one seems interesting in the case of the transient events. More information here.

warnings filter reset to default within package

There are several cases where the warnings filter is reset to the default. As far as I can tell, this prohibits a user from using their own warning filters. For example, the following still produces DeprecationWarnings

import warnings

moc = MOC()
with warnings.catch_warnings():
    warnings.simplefilter("ignore")
    boundaries = moc.get_boundaries()

I suspect that the internal use of warnings.simplefilter("default") is making the above ineffective. Would it be possible to have these default filters removed? Happy to do so and submit a PR if necessary.

STMOC missing attributes _to_json and _to_str

When attempting to serialize an STMOC, json and str representations are not yet available:
AttributeError: 'STMOC' object has no attribute '_to_json'
AttributeError: 'STMOC' object has no attribute '_to_str'

Add read/write support for ASCII serialization

See section 2.3.2 String serialization of document http://ivoa.net/documents/MOC/20190215/WD-MOC-1.1-20190215.pdf for details

MOC.from_polygon() returns empty MOC for a square that should have an area of 48 deg2

mocpy.MOC.from_polygon is returning an empty MOC for a simple square polygon that should have an area of about 48 deg2.

Here's the example:

>>> from astropy import units as u
>>> from mocpy import MOC
>>> import numpy as np
>>> ra = [174.75937396073138, 185.24062603926856, 184.63292896369916, 175.3670710363009]
>>> dec = [-49.16744206799886, -49.16744206799887, -42.32049830486584, -42.32049830486584]
>>> MOC.from_polygon(ra * u.deg, dec * u.deg)
[]

What is very, very strange, is that by adding a tiny value of 1e-13 to the coordinates, I get a non-empty and essentially correct result!

>>> MOC.from_polygon((np.asarray(ra) + 1e-13) * u.deg, dec * u.deg)
[[2685120919604559872 2685121744238280704]
 [2685125317651070976 2685126142284791808]
 [2685126417162698752 2685130540331302912]
 [2685218776139431936 2685221250040594432]
 [2685221799796408320 2685222349552222208]
 [2685222899308036096 2685227297354547200]
 [2685400195558014976 2685401020191735808]
 [2685401295069642752 2685403219214991360]
 [2685404593604526080 2685405418238246912]
 [2685405693116153856 2685412015308013568]
 [2685471114058006528 2685471388935913472]
 [2685472213569634304 2685473587959169024]
 [2685475512104517632 2685475786982424576]
 [2685476611616145408 2685482384052191232]
 [2685488156488237056 2685488981121957888]
 [2685489255999864832 2685491180145213440]
 [2685492554534748160 2685492829412655104]
 [2685493104290562048 2685493379168468992]
 [2685494203802189824 2685552752796368896]
 [2685558525232414720 2685559349866135552]
 [2685559624744042496 2685561548889391104]
 [2685562923278925824 2685563747912646656]
 [2685564022790553600 2685834227773079552]
 [2685840549964939264 2685840824842846208]
 [2685841649476567040 2685843023866101760]
 [2685844948011450368 2685845222889357312]
 [2685846322400985088 2685869412145168384]
 [2685869687023075328 2685983761354457088]
 [2685992557447479296 2686001353540501504]
 [2686045334005612544 2686054130098634752]
 [2686062926191656960 2686071722284679168]
 [2686115702749790208 2686265236331167744]
 [2686274032424189952 2686282828517212160]
 [2686326808982323200 2686335605075345408]
 [2686344401168367616 2686353197261389824]
 [2688648977540186112 2689361461074984960]
 [2689370257168007168 2689379053261029376]
 [2689423033726140416 2689431829819162624]
 [2689440625912184832 2689449422005207040]
 [2689493402470318080 2689642936051695616]
 [2689651732144717824 2689661077993553920]
 [2689661627749367808 2689662177505181696]
 [2689664926284251136 2689665476040065024]
 [2689666025795878912 2689666575551692800]
 [2689704508702851072 2689713854551687168]
 [2689714404307501056 2689714954063314944]
 [2689717702842384384 2689718252598198272]
 [2689718802354012160 2689719352109826048]
 [2689722100888895488 2689731446737731584]
 [2689731996493545472 2689732546249359360]
 [2689735295028428800 2689735844784242688]
 [2689736394540056576 2689736944295870464]
 [2689774877447028736 2690020343417929728]
 [2690021168051650560 2690037935603974144]
 [2690038760237694976 2690042333650485248]
 [2690043158284206080 2690043433162113024]
 [2690047556330717184 2690047831208624128]
 [2690056352423739392 2690089062894665728]
 [2690089337772572672 2690090162406293504]
 [2690091536795828224 2690108304348151808]
 [2690109128981872640 2690111053127221248]
 [2690111328005128192 2690112427516755968]
 [2690117925074894848 2690118199952801792]
 [2690126721167917056 2690128645313265664]
 [2690128920191172608 2690129744824893440]
 [2690197089912094720 2690200663324884992]
 [2690201487958605824 2690201762836512768]
 [2690205886005116928 2690206160883023872]
 [2690337827400450048 2690372187138818048]
 [2690373011772538880 2690390329080676352]
 [2690390603958583296 2690394727127187456]
 [2690395002005094400 2690395826638815232]
 [2690399400051605504 2690400224685326336]
 [2690408196144627712 2690411769557417984]
 [2690412594191138816 2690412869069045760]
 [2690416992237649920 2690417267115556864]
 [2690478564888805376 2690482688057409536]
 [2690482962935316480 2690483787569037312]
 [2690487360981827584 2690487910737641472]
 [3063493657047859200 3063494481681580032]
 [3063498055094370304 3063498879728091136]
 [3063499154605998080 3063503277774602240]
 [3063538737024598016 3063539836536225792]
 [3063540111414132736 3063540386292039680]
 [3063540661169946624 3063542035559481344]
 [3063542310437388288 3063542585315295232]
 [3063547258239713280 3063551656286224384]
 [3064829013919793152 3064829288797700096]
 [3064830113431420928 3064831487820955648]
 [3064833411966304256 3064833686844211200]
 [3064834511477932032 3064840283913977856]
 [3064898832908156928 3064899657541877760]
 [3064899932419784704 3064901856565133312]
 [3064903230954668032 3064904055588388864]
 [3064904330466295808 3064910652658155520]
 [3064916974850015232 3064917249727922176]
 [3064917799483736064 3064919448751177728]
 [3064920823140712448 3064921647774433280]
 [3064921922652340224 3064981021402333184]
 [3064988443105820672 3064989817495355392]
 [3064991741640704000 3064992016518610944]
 [3064992841152331776 3064998613588377600]
 [3064998888466284544 3065055788193021952]
 [3065060186239533056 3065064584286044160]
 [3065086574518599680 3065090972565110784]
 [3065095370611621888 3065099768658132992]
 [3065121758890688512 3065196525681377280]
 [3065200923727888384 3065205321774399488]
 [3065227312006955008 3065231710053466112]
 [3065236108099977216 3065240506146488320]
 [3065268268815089664 3065269093448810496]
 [3065269368326717440 3065271292472066048]
 [3065272666861600768 3065273491495321600]
 [3065273766373228544 3065618738146443264]
 [3065623136192954368 3065627534239465472]
 [3065649524472020992 3065653922518532096]
 [3065658320565043200 3065662718611554304]
 [3065684708844109824 3065759475634798592]
 [3065763873681309696 3065768271727820800]
 [3065790261960376320 3065794660006887424]
 [3065799058053398528 3065803456099909632]
 [3071454945866678272 3071811187634077696]
 [3071815585680588800 3071819983727099904]
 [3071841973959655424 3071846372006166528]
 [3071850770052677632 3071855168099188736]
 [3071877158331744256 3071951925122433024]
 [3071956323168944128 3071960721215455232]
 [3071982711448010752 3071987109494521856]
 [3071991507541032960 3071995905587544064]
 [3072017895820099584 3072374137587499008]
 [3072378535634010112 3072382933680521216]
 [3072404923913076736 3072409596837494784]
 [3072409871715401728 3072410146593308672]
 [3072411520982843392 3072411795860750336]
 [3072412070738657280 3072412345616564224]
 [3072413720006098944 3072418392930516992]
 [3072418667808423936 3072418942686330880]
 [3072420317075865600 3072420591953772544]
 [3072420866831679488 3072421141709586432]
 [3072440108285165568 3072515149953761280]
 [3072515424831668224 3072515699709575168]
 [3072517074099109888 3072517348977016832]
 [3072517623854923776 3072517898732830720]
 [3072519273122365440 3072523946046783488]
 [3072524220924690432 3072524495802597376]
 [3072525870192132096 3072526145070039040]
 [3072526419947945984 3072526694825852928]
 [3072545661401432064 3072550334325850112]
 [3072550609203757056 3072550884081664000]
 [3072552258471198720 3072552533349105664]
 [3072552808227012608 3072553083104919552]
 [3072554457494454272 3072559130418872320]
 [3072559405296779264 3072559680174686208]
 [3072561054564220928 3072561329442127872]
 [3072561604320034816 3072561879197941760]
 [3073706745680363520 3073952211651264512]
 [3073953036284985344 3073969803837308928]
 [3073970628471029760 3073974201883820032]
 [3073975026517540864 3073975301395447808]
 [3073979424564051968 3073979699441958912]
 [3073988220657074176 3074023130151256064]
 [3074023405029163008 3074040172581486592]
 [3074040997215207424 3074045120383811584]
 [3074045395261718528 3074046219895439360]
 [3074049793308229632 3074050617941950464]
 [3074058589401251840 3074062712569856000]
 [3074062987447762944 3074063262325669888]
 [3074063537203576832 3074063812081483776]
 [3074067385494274048 3074068210127994880]
 [3074128958145429504 3074132531558219776]
 [3074133356191940608 3074133631069847552]
 [3074137754238451712 3074138029116358656]
 [3074269695633784832 3074304055372152832]
 [3074304880005873664 3074319998290755584]
 [3074320273168662528 3074321097802383360]
 [3074322472191918080 3074325495848894464]
 [3074326870238429184 3074327145116336128]
 [3074340064377962496 3074343637790752768]
 [3074344462424473600 3074344737302380544]
 [3074348860470984704 3074349135348891648]
 [3074410433122140160 3074412357267488768]
 [3074412632145395712 3074413456779116544]]

Add support to release linux aarch64 wheels

Problem

On aarch64, pip install mocpy builds the wheels from source code and then installs it. It requires the user to have a development environment installed on their system. Also, it takes more time to build the wheels than downloading and extracting the wheels from PyPI.

Resolution

On aarch64, pip install mocpy should download the wheels from PyPI.

@bmatthieu3, Please let me know your interest in releasing aarch64 wheels. I can help with this.

Notebooks for time coverage are missing

The notebooks for time MOC (https://github.com/cds-astro/mocpy/blob/ranges2D/notebooks/tmoc.ipynb) and space-time MOC (https://github.com/cds-astro/mocpy/blob/ranges2D/notebooks/Space%20%26%20Time%20coverages.ipynb) return 404. So either the URLs are not correct, or they are not world-readable.

from_table method missed in the last version

In the last mocpy version the method from_table is supressed. Why?

from_healpix_cells documentation

It might be helpful to mention in the documentation that only the 'nested' healpix ordering can be used with this method -- particularly as astropy.healpix defaults to 'ring' ordering. (Also maybe that the 'depth' of mocpy corresponds to the 'level' of astropy.healpix.)

Plotiing a MOC without giving a center.

Plotting MOCs as shown in the doc examples is a great feature very helpful for e.g. tuning MOC generation.
This unfortunately requires the user to set by hand the projection center.
This plotting feature would be quite more convenient whether that center was computed automatically from the MOC itself.

The code snippet below, derived from the doc, shows how such feature could be implemented

from mocpy import MOC, World2ScreenMPL
from astropy.coordinates import Angle, SkyCoord
import astropy.units as u
# Load a MOC
filename = './../../resources/P-SDSS9-r.fits'
moc = MOC.from_fits(filename)
# Plot the MOC using matplotlib
import matplotlib.pyplot as plt
fig = plt.figure(111, figsize=(15, 10))
# Define a astropy WCS easily
with World2ScreenMPL(fig, 
        fov=200 * u.deg,
        #center=SkyCoord(0, 20, unit='deg', frame='icrs'),
        center=moc.center(unit='deg', frame='icrs'), ## <<< Proposal
        coordsys="icrs",
        rotation=Angle(0, u.degree),
        projection="AIT") as wcs:
    ax = fig.add_subplot(1, 1, 1, projection=wcs)
    # Call fill with a matplotlib axe and the `~astropy.wcs.WCS` wcs object.
    moc.fill(ax=ax, wcs=wcs, alpha=0.5, fill=True, color="green")
    moc.border(ax=ax, wcs=wcs, alpha=0.5, color="black")
plt.xlabel('ra')
plt.ylabel('dec')
plt.title('Coverage of P-SDSS9-r')
plt.grid(color="black", linestyle="dotted")
plt.show()

Add support for experimental "T-MOC" (temporal MOC)

CDS has started to implement and use a prototype format called T-MOC (temporal MOC) aimed at providing the temporal counterpart of a MOC, ie the temporal coverage of a survey.
Use cases include visualisation of the time covered by a survey, fast search of data in a given time range, comparison and intersection of time coverage between different surveys.

@bmatthieu3 could implement in a new branch the creation of a T-MOC in MOCPy, from a list of either:

t_start and t_end
t_start and duration

t_start and t_end could be expressed as astropy.time.Time instances

MOC creation from pixel range

It could be useful a new method able to create a MOC from a specific pixel range as implemented in the Aladin Desktop.

best,
Giuseppe

Create the MOC for a set of FITS images

It would be nice if one could create the MOC corresponding to a set of FITS images (having WCS information)

Buiiding MOC from images located in ant FITS extension

The current API allows to build a MOCs from a set of FITS files located in a given folder:

MOC.from_fits_images(path_l, max_norder)

It is however not possible to use the same convenient method when the images are in a FITS extension. This is a case e.g. when we want to build a MOC from exposure maps which are usually not in the primary extension.
In this case we have to build MOCs for each individual file by using MOC.from_fits_image(hdu, max_norder, mask=None) and to merge them. This is a bit boring.

Having a function like this:

MOC.from_fits_dhu(path_l, hdu, max_norder)

Would be very helpful.

publish wheels for pyodide

Right now, MOCpy cannot be installed via micropip. To solve that we could try to follow this short few lines at the bottom of this file :
https://github.com/pyodide/pyodide/blob/main/docs/development/new-packages.md

Broken TimeMOC.from_time_ranges() ?

When I try to create a TimeMOC from a single time interval, an error is raised:

~/anaconda3/envs/mocpy/lib/python3.7/site-packages/mocpy/tmoc/tmoc.py in from_time_ranges(cls, min_times, max_times, delta_t)
     76         depth = TimeMOC.time_resolution_to_order(delta_t)
     77         intervals = core.from_time_ranges(
---> 78             min_times.jd.astype(np.float64),
     79             max_times.jd.astype(np.float64),
     80         )

AttributeError: 'float' object has no attribute 'astype'

The .jd property of a single element Time object returns a float value, not an array. So the .astype() method is not applicable.

Consider renaming WCS class

The following file:

https://github.com/cds-astro/mocpy/blob/master/mocpy/moc/plot/wcs.py

defines a WCS class but this confused me at first in the docs given that the core astropy package also includes a class of the same name. I wonder if it could be renamed in MOCPy since it's just a high-level convenience not a general WCS class? (it could even be a function context manager rather than a class?)

Add python 3.10 wheels

Could you please add 3.10 wheels to pypi? That would let us avoid building the wheel from source in downstream CI testing.

STMOC from existing MOC

We should consider having a method to enhance an existing MOC with time information so as to produce with STMOC. The example I have in mind is a set of MOC I've been creating for TESS footprints. I would like to just take my existing MOC and add a time range for it.

Overplotting a MOC on a custom WCS projected figure with MOC.fill()

Hello,

I was trying to add MOC overlays to my all-sky plots using the MOC.fill() method but the resulting graph seems to have its axes modified.

I can try to give a minimal example, using a custom WCS object to define a matplotlib subplot projection.
Then, I create a MOC object and overplot it using moc.fill(ax=ax, wcs=wcs).

import matplotlib.pyplot as plt
from astropy.wcs import WCS
import numpy as np
import healpy as hp
from mocpy import MOC
import astropy.units as u

# Generate an abritrary WCS object
wcs = WCS(naxis=2)
wcs.wcs.ctype = ["RA---AIT", "DEC--AIT"] 
wcs.wcs.crval = [0.0, 0.0]
wcs.wcs.cdelt = np.array([-0.675, 0.675])
wcs.wcs.crpix = [240.5, 120.5]

# Get HEALPix cell centers
nside = 64
ra, dec = hp.pix2ang(
    nside,
    np.arange(hp.nside2npix(nside)),
    nest=False,
    lonlat=True
)

# MPL figure with WCS projection
fig = plt.figure(figsize=(15, 7))
ax = plt.subplot(
    projection=wcs
)

# Plot the HEALPix centers (just to test)
ax.scatter(ra, dec, 0.1, transform=ax.get_transform("world"))
ax.grid()

# Define a MOC union
moc1 = MOC.from_cone(
    lon=60*u.deg,
    lat=60*u.deg,
    radius=20*u.deg,
    max_depth=10,
    delta_depth=2
)
moc2 = MOC.from_cone(
    lon=0*u.deg,
    lat=0*u.deg,
    radius=10*u.deg,
    max_depth=10,
    delta_depth=2
)
moc = moc1.union(moc2)

# Plot the MOC in the current figure
moc.fill(ax=ax, wcs=wcs)

This is what I get:

The inner plot seems right, however the x_lim and y_lim are modified by mocpy/mocpy/moc/plot/axis_viewport.py.

If I comment:

mocpy/mocpy/moc/plot/fill.py

Line 86 in 48dbeef

axis_viewport.set(ax, wcs)

I get what I expect:

Am I doing something wrong?
If not, would it be possible to add an option to the plotting methods to deactivate the call to axis_viewport please?

Many thanks,

(Tested on Python 3.7, mocpy 0.10.0)

Mocpy vs. pymoc

Hi,

There is another MOC package -- pymoc. From my Debian stats I would estimate that pymoc is one of the regularly used python tools, and the "used-by" buttom of github refers to some use in the Herschel Extragalactic Legacy Project. It seems, however, no longer developed (stalled since 2017).
Are there any relations between those two packages? Attempts to unify here? It would be a pity if there were two competing packages for the same thing.

Cc @grahambell (pymoc), @raphaelshirley (H-E-L-P)

(I am a developer of the Debian Astro Pure Blend and just stumbled upon astropy/astropy.github.com#319)

Add travis-ci?

How about setting up travis-ci for this repo, with a Python 2.7 and 3.5 build?

If you want I can send a PR with a .travis.yml file, but activating travis-ci needs to be done by an owner first (it's free and quick).

Test notebookbooks in CI

How about executing the notebooks in travis-ci to make sure they don't break again?

(see #17, #18, #19)

E.g. with https://github.com/paulgb/runipy you can do:

runipy notebooks/MOCPy_example1.ipynb

or I think by now the notebook execute is built in with Jupyter directly:
http://nbconvert.readthedocs.io/en/latest/execute_api.html
E.g. one could have a test_notebooks.py that does a glob to get all notebooks and then executes them and if an error is present exists with non-zero status code and thus breaks the CI build.

Example 5 notebook broken with Python 3

For https://github.com/cds-astro/mocpy/blob/master/notebooks/MOCPy_example5.ipynb with Python 3 I and this cell

#table = moc.query_simbad(10000)
table = moc.query_vizier_table('I/239/hip_main', max_rows=100000)
print(table)

I get this error:

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-4-ded2ceed93ca> in <module>()
      1 #table = moc.query_simbad(10000)
----> 2 table = moc.query_vizier_table('I/239/hip_main', max_rows=100000)
      3 print(table)

~/software/anaconda3/envs/hgps-hips/lib/python3.6/site-packages/mocpy/moc.py in query_vizier_table(self, table_id, max_rows)
    377         """
    378 
--> 379         return self._query(table_id, max_rows)
    380 
    381     def _query(self, resource_id, max_rows):

~/software/anaconda3/envs/hgps-hips/lib/python3.6/site-packages/mocpy/moc.py in _query(self, resource_id, max_rows)
    408 
    409         from astropy.io.votable import parse_single_table
--> 410         table = parse_single_table(tmp_vot).to_table()
    411 
    412         # finally delete temp files

~/software/anaconda3/envs/hgps-hips/lib/python3.6/site-packages/astropy/io/votable/table.py in parse_single_table(source, **kwargs)
    153         kwargs['table_number'] = 0
    154 
--> 155     votable = parse(source, **kwargs)
    156 
    157     return votable.get_first_table()

~/software/anaconda3/envs/hgps-hips/lib/python3.6/site-packages/astropy/io/votable/table.py in parse(source, columns, invalid, pedantic, chunk_size, table_number, table_id, filename, unit_format, datatype_mapping, _debug_python_based_parser)
    135             _debug_python_based_parser=_debug_python_based_parser) as iterator:
    136         return tree.VOTableFile(
--> 137             config=config, pos=(1, 1)).parse(iterator, config)
    138 
    139 

~/software/anaconda3/envs/hgps-hips/lib/python3.6/site-packages/astropy/io/votable/tree.py in parse(self, iterator, config)
   3339         config['_current_table_number'] = 0
   3340 
-> 3341         for start, tag, data, pos in iterator:
   3342             if start:
   3343                 if tag == 'xml':

ValueError: 1:0: syntax error

@bmatthieu3 or @tboch - Can you reproduce?

( I didn't try with Python 2, don't know if it works there or not )

python +, -, & operators redefinition for union, sub, intersect between mocs

pymoc has a nice python API for its logical operations between mocs i.e. Python +, -, ... operators are redefined.
cf: the online pymoc doc

I personaly think this is a very nice feature and may be great to have it in mocpy.

Empty MOC has interval set array with wrong shape

An empty MOC has an interval set with the wrong shape. The shape should be (0, 2), but it is (1, 0).

>>> from mocpy import MOC
>>> MOC()._interval_set.nested.shape
(1, 0)

Comment on slowness of 'from-skycoords' function

@tboch you say in a comment in 'from_coo_list' the loop over SkyCoords is "very slow". I didn't check this particular case in details, but I have crossed such issue in the past; may be useful to you.

Itemize operations with SkyCoords arrays are very slow. Performance may be significantly increased by reading the values out from the SkyCoord array (in the desired unit, 'deg' for instance) to a simpler structure (e.g, numpy array (N,2)). And then, even if not vectorizing the "add_position" call, it should perform better.

Thanks.

Increase range2uniq performances (x10?)

Using the algo in HpxRange and MOCIteratorFromRanges in cdshealpix-rust

networkx.connected_component_subgraphs removed

The function networkx.connected_component_subgraphs has been removed from the latest version of networkx.

I think either the version of networkx for this package should be pinned to <=2.3, or the calls to the above function should be replaced with the suggested replacement, e.g.

G_subgraphs = (G.subgraph(c).copy() for c in connected_components(G))

Happy to submit a PR for this if necessary after some guidance on which would be preferred.

Create conda recipe in conda-forge

TimeMOC resolution issue

The Time objects of astropy.time are coded with 2 double-precision floating point numbers. This allows to have access to microsecond resolution.

Since mocpy is using astropy.time, we should have access to microsecond resolution (the deepest TimeMOC order), but we don't.

Simple code to test this:

>>> import mocpy
>>> from astropy.time import Time
>>> tmin = Time(['2000-01-01T00:00:00.000000Z','2000-01-01T00:00:00.000010Z'])
>>> tmax = Time(['2000-01-01T00:00:00.000005Z','2000-01-01T00:00:00.000015Z'])
>>> (tmax-tmin).sec
array([5.00000041e-06, 5.00000041e-06])
>>> tm = mocpy.TimeMOC.from_time_ranges(tmin,tmax)
>>> tm
[[211813443995435008 211813445069176832]]
>>> from astropy.time import TimeDelta
>>> tm = mocpy.TimeMOC.from_time_ranges(tmin,tmax,delta_t=TimeDelta(1e-6,format='sec'))
>>> tm
[[211813444800000000 211813444800000001]]

We shouls have 2 intervals lasting 5 µs each, but we get only one. Internally, the code is converting input Time objects into jd (see here for instance), and this destroys the microsecond resolution.

from mocpy import MOC
m1 = MOC.from_file('demo-data/P-SDSS9-r.fits')
m1.plot(title='SDSS9 band r coverage', coord='C')

seems to be broken:

AttributeError                            Traceback (most recent call last)
<ipython-input-2-31a14f640c85> in <module>()
      1 from mocpy import MOC
----> 2 m1 = MOC.from_file('demo-data/P-SDSS9-r.fits')
      3 m1.plot(title='SDSS9 band r coverage', coord='C')

AttributeError: type object 'MOC' has no attribute 'from_file'

I think the same call is made in other notebooks.

cds-astro / mocpy Goto Github PK

mocpy's People

Stargazers

Watchers

Forkers

mocpy's Issues

In [2]: mc = MOC.from_file('/Users/haussel/Downloads/Herschel-Stripe-82_MOC.fits')

Problem

Resolution

Recommend Projects

Recommend Topics

Recommend Org