low-sky / pyprops Goto Github PK
View Code? Open in Web Editor NEWThis project forked from akleroy/pyprops
CPROPS python implementation
This project forked from akleroy/pyprops
CPROPS python implementation
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Python "CPROPStoo" Implementation -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Implementation of the signal-identification, object extraction, and emission characterization package "cprops" in python. -------------------- OVERALL STRUCTURE -------------------- cube - a generic CASA/FITS interface to pull in a data cube data - read and manipulate data cubes. Inherits cube. mask - create, manipulate, and read/write masks. Inherits cube. noise - defines a class to derive noise estimates assign - a "colored" cube that assigns pixels to objects lmax - identifies local maxima struct - helper class to build morphological structuring elements -------------------- NEXT STEPS -------------------- - Beam handling in "cube" - Unit handling in "data" (Jy <-> K main hurdle, leverage astropy) - Automatic masking parameters from negative (accept a thresholding definition and suggest a value that yields no negatives). - Assignment object - "Moment" object - Up noise dimensionality - Noise read/write - Revisit "struct" and make it cleaner/clearer - "Suggested subcubes" in mask - Primary beam correction into "cube" (and noise?) - Rework noise so that it inherits cube? Noise needs to decide if it is something that lives on its own or is always recomputed on the fly. I like the former. - a general validate() function that runs a suite of sanity checks and reports back. Cleaning up some of this stuff is out of scope but we can at least shout on units, untyped axes, axis order, etc. -------------------- OTHER DIRECTIONS -------------------- - all kinds of error checking - moment creation / linkage to CASA / error propagation - radial profile construction - two-d header / spectrum header extraction - useful for writing a minimum sized noise object. - spectral stacking implementation -------------------- COMMENTS -------------------- - Basically, every cube operation for something the size of the Antennae Cycle 0 on a reasonable desktop takes about 1-2s for a single numpy cube operation. A typical "astronomical" operation involves say 3-10 of these and sometimes ~100. That sets your timescales for anything. - We may want a "blank value" for the cube object (in addition to the validity flag). Masked arrays are still a mixed bag and adding a blanking step to the beginning of each task makes things slow. Similarly, pre-computing the min and max would probably be useful. My rough checks are that doing this prep within each program takes something like 5-6 seconds, which adds up. Alternatively, we may want a high level routine in cube or data object to expose a numpy-friendly object with a blank value filled in. Issue is that what we want is not obvious (its 0 for summing, it's a very low or high value for masking). - The merge calculation is slow. Not prohibitively, but N labelings of a big cube are heading in that direction. The issue is at least partially that the whole cube is thrown in and repeatedly processed in operations where only a tiny fraction of the data are interesting. Right now it's ~2 seconds per label operation on the Antennae, so ~3 minutes to get the matrix for 100 levels. Astrodendro (see below) offers one path: loop over the individual sorted pixels. - Easy link to astrodendro: cube + mask + valid to set invalid values to floor, then pass the manipulated cube in. It's fast when heavily downselected first, which makes sense. Twiddle a minute on the Antennae with 3-sigma clipping (~1% of the data). However, it's slow (worse than merging) when fed the whole cube, so a clip first is going to be needed.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.