lgrcia / prose Goto Github PK
View Code? Open in Web Editor NEWModular image processing pipelines with Python. Built for Astronomy.
Home Page: https://prose.readthedocs.io
License: MIT License
Modular image processing pipelines with Python. Built for Astronomy.
Home Page: https://prose.readthedocs.io
License: MIT License
'
character in absoulte paths of files breaks some of the behaviours of prose.FitsManager
When loading telescope (e.g. loading a .phot), error when a root directory has same name has telescope ('cause Telescope.load first check a file with telescope name in current folder in case of a custom telescope file)
Set image plots axis with physical limits like RA and DEC range, or arc-seconds. Useful to compare images of different resolutions
In prose/observation.py
line 89, it is written:
"ERROR": self.error
But this creates an eror as it should be:
"ERROR": self.diff_error
Handle failing twirl in calibration
Error we got is in blocks/registration.py
line 354
i, j = tutils.cross_match(new_ref, s, tolerance=tolerance, return_ixds=True).T
got
ValueError: not enough values to unpack (expected 2, got 0)
The longitude of the site add an extra minus sign which was creating some errors in the computation of BJT_TDB times.
Must change in /.prose/liverpool.telescope
:
latlong:
- 28.7624
- -17.8792
into:
latlong:
- 28.7624
- 17.8792
Probably no stars found on stack:
File ".../prose/prose/pipeline/photometry.py", line 94, in run
self.detection_s.run(show_progress=False)
...
File ".../prose/prose/blocks/detection.py", line 91, in single_detection
coordinates = np.transpose(np.array([sources["xcentroid"].data, sources["ycentroid"].data]))
TypeError: 'NoneType' object is not subscriptable
example on 20180826 SSO/Io
Using obs._compute_psf_model()
or obs.plot_psf_model()
gives inverted values of obs.stack.fwhmx
and obs.stack.fwhmy
.
In pipeline.Calibration
, if an image is discarded by blocks.AffineTransform
it is not retained in the final blocks.Xarray
but is still saved reduced (blocks.SaveReduced
appears before). Hence the pipeline.Photometry
will fail (taking from the calibration xarray which has less images than the number reduced ones).
Possible solutions:
Put a limit on filled region areas
Write guidelines on the parameters to use when doing plate solving.
There is a trade-off between the number of stars to consider against the number of queried stars to match (more stars make the match slower). For example, we retain the n
brightest Gaia stars in a field s
times larger than the expected field of view (using image size and pixel known plate scale), and we compare these stars to the m
brightest detected stars in the image. All these parameters play an important role, especially since brightness of Gaia stars is sorted from phot_g_mean_fluxes
that may not match the relative brightness of the stars in a given observation with different wavelength sensitivity.
Some guidelines would be useful (see #58)
Develop more systematic testings (per block and global) and switch the testing framework to pytest
It would be nice to be able to exclude certain comparison stars, either because they look blended or because they are higly variable etc.
Somthing like:
df = obs.broeg2005(exclude_comps = [1,56,86])
Add an EPSF Block
and a espf
kwarg in the PSF fitting blocks so that they are applied on the epsf, otherwise on all cutouts stored in blocks.cutouts
Even if i set calibration = False
reduction is failling if calibrations images are missing.
An attribute in classes that turns False
(unsafe) when a set of stars coordinates has been used by a block (for example to extract photometry) but is modified by another block.
A possible behaviour would be:
s = Sequence([
blocks.StarsDetection(),
blocks.Photometry(),
blocks.LimitStars(min=5)
], stars_safe=True) # True by default
Warning: stars_coords have been modified after beeing used in block `Photometry`
Through a Sequence._stars_safe
updated after every Image
is ran
Typo in version= "eastman in Observation.compute_bjd()
, lines 301-302
current code:
elif version == "eastman":
bjd_time = utils.jd_to_bjd(self.jd_utc + exposure_days/2, self.stack.skycoord.ra.deg, self.skycoord.dec.deg)
should be:
elif version == "eastman":
bjd_time = utils.jd_to_bjd(self.jd_utc + exposure_days/2, self.stack.skycoord.ra.deg, self.stack.skycoord.dec.deg)
Add a date to the telescope definition so that definiton is date dependant:
Telescope({...}, date="2018-12-01")
For example if an image is taken on the 2018-12-13 and the following telescopes are available:
2018-11-12_name.telescope
2018-12-01_name.telscope
2022-10-01_name.telescope
2018-12-01_name.telscope
is chosen, as the latest past telescope w.r.t image date
function .save_mcmc_file
is not documented
In some cases, the Broeg2005() gives "two" comparison stars as the best ones for differential photometry. One of the two stars presents some variations exactly during the transit time, which may have an impact one the transit signal. Is there any option to put that star out of Broeg2005() consideration?
Thanks in advance ;)
Add PANSTARS image in findercharts
(e.g. https://ps1images.stsci.edu/cgi-bin/ps1cutouts?pos=--&filter=z&filetypes=stack&auxiliary=data&size=1200&output_size=0&verbose=0&autoscale=99.500000&catlist=)
When I run Plate solving tutorial follow document, I got an error.
Here is my code:
from prose import Sequence, blocks
plate = Sequence([
blocks.detection.SegmentedPeaks(min_separation=15, n_stars=15),
blocks.catalogs.PlateSolve(debug=True)
])
plate.run(image, show_progress=False)
and the error:
Output exceeds the [size limit](command:workbench.action.openSettings?%5B%22notebook.output.textLineLimit%22%5D). Open the full output data [in a text editor](command:workbench.action.openLargeOutput?fe124561-7179-4dfa-8328-049ad16dff0c)
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
[e:\03_Projects\ImageProcess\imlt\prose\PlatSolving.ipynb](file:///E:/03_Projects/ImageProcess/imlt/prose/PlatSolving.ipynb) Cell 14 in ()
[1](vscode-notebook-cell:/e%3A/03_Projects/ImageProcess/imlt/prose/PlatSolving.ipynb#X16sZmlsZQ%3D%3D?line=0) from prose import Sequence, blocks
[3](vscode-notebook-cell:/e%3A/03_Projects/ImageProcess/imlt/prose/PlatSolving.ipynb#X16sZmlsZQ%3D%3D?line=2) plate = Sequence([
[4](vscode-notebook-cell:/e%3A/03_Projects/ImageProcess/imlt/prose/PlatSolving.ipynb#X16sZmlsZQ%3D%3D?line=3) blocks.detection.SegmentedPeaks(min_separation=15, n_stars=15),
[5](vscode-notebook-cell:/e%3A/03_Projects/ImageProcess/imlt/prose/PlatSolving.ipynb#X16sZmlsZQ%3D%3D?line=4) blocks.catalogs.PlateSolve(debug=True)
[6](vscode-notebook-cell:/e%3A/03_Projects/ImageProcess/imlt/prose/PlatSolving.ipynb#X16sZmlsZQ%3D%3D?line=5) ])
----> [8](vscode-notebook-cell:/e%3A/03_Projects/ImageProcess/imlt/prose/PlatSolving.ipynb#X16sZmlsZQ%3D%3D?line=7) plate.run(image, show_progress=False)
File [c:\Users\Alfre\AppData\Local\Programs\Python\Python310\lib\site-packages\prose\core\sequence.py:103](file:///C:/Users/Alfre/AppData/Local/Programs/Python/Python310/lib/site-packages/prose/core/sequence.py:103), in Sequence.run(self, images, terminate, show_progress, loader)
101 self.n_processed_images = 0
102 self.discards = {}
--> 103 self._run(loader=loader)
105 if terminate:
106 self.terminate()
File [c:\Users\Alfre\AppData\Local\Programs\Python\Python310\lib\site-packages\prose\core\sequence.py:119](file:///C:/Users/Alfre/AppData/Local/Programs/Python/Python310/lib/site-packages/prose/core/sequence.py:119), in Sequence._run(self, loader)
116 image.i = i
118 for block in self.blocks:
--> 119 block._run(image)
120 # This allows to discard image in any Block
121 if image.discard:
File [c:\Users\Alfre\AppData\Local\Programs\Python\Python310\lib\site-packages\prose\core\block.py:54](file:///C:/Users/Alfre/AppData/Local/Programs/Python/Python310/lib/site-packages/prose/core/block.py:54), in Block._run(self, *args, **kwargs)
...
File [c:\Users\Alfre\AppData\Local\Programs\Python\Python310\lib\site-packages\prose\blocks\registration.py:10](file:///C:/Users/Alfre/AppData/Local/Programs/Python/Python310/lib/site-packages/prose/blocks/registration.py:10), in distance(p1, p2)
9 def distance(p1, p2):
---> 10 return np.sqrt(np.power(p1[0] - p2[0], 2) + np.power(p1[1] - p2[1], 2))
TypeError: 'PointSource' object is not subscriptable
My system is Win11 22624.1391, Python version is 3.10.9, prose version is 2.3.0.
Setup an automatic release pipeline for Gihtub and pypi
In prose/pipeline/calibration.py, the file saved to stack_path is self.reference and not self.stack. The FitsManager was not recognizing the stack image because it said "Light frame" in the header.
I think it would be a nice idea if there would be a way to see the RA and DEC for the selected star in the summary plots. At the moment the RA and DEC on the summary plots are those from the centre of the image.
I see the following block of code (lines 1011-1018) in observation.py
params = [ ("Telescope", f"{image.telescope.name}"), ("Date", f"{image.night_date}"), ("Filter", f"{image.filter}"), ("Exposure", f"{image.exposure}"), ("RA", f"{image.skycoord.ra:.4f}"), ("DEC", f"{image.skycoord.dec:.4f}"), ("Dimenion", f"{image.shape[0]}x{image.shape[1]} pixels"),
]
I think it would be good to have the option to change the RA and DEC line from skycoord.ra
and skycoord.dec
to something like stars_coords.ra
and stars_coords.dec
Many thanks,
Alex
To cite packages used in blocks and sequences in a correct way.
Deal with the case where reference image in Calibration is bad, e.g. leading to:
Warning: image 0 discarded in SegmentedPeaks
Could think about a richer view, like zoomable stack
It seems that Jaxlib (required to install prose 3.0.0) is not supported by Windows ( cfr https://stackoverflow.com/questions/62585395/not-able-to-install-jaxlib )
Possible solution: https://github.com/cloudhan/jax-windows-builder
Case studies including:
I got the following error during the reduction :
RUN: Reduction: 18%|███▊ | 199/1096 [01:27<06:24, 2.33files/s]WARNING: File may have been truncated: actual file length (2018364) is smaller than the expected size (8562240) [astropy.io.fits.file]
TypeError: buffer is too small for requested array
when I open the corresponding file I get the following error in ds9: a SIGBUS or SIGSEGV error has been received
It looks like some files are corrupted.
function to generate url to exoFOP page of a tic
When the TELESCOP keyword in the FITS header contains a special character, eg: EQMOD ASCOM HEQ5/6, prose fails. The workaround I have found for this is editing the TELESCOP field to another field (in my case SITENAME) in following lines:
Line 83 in IO.PY - telescope_kw="SITENAME"
Line 118 in IMAGE.PY - self.header.get("SITENAME", "")
It would be good if prose could pull the keyword designed for keyword_telescope: SITENAME
or name:
from the .telescope files
Thanks a lot
Alex
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.