Comments (8)
I always do the following
flist=glob(pattern)
flist.sort()
note sort is in place, so this will not work, since sort() returns None
flist=glob(pattern).sort()
from piff.
I was clearly confused
from piff.
Or
flist = sorted(glob(pattern))
But the point remains that it would be helpful to have other ways to specify the list of image and catalog files to make sure they match up 1-1.
from piff.
FYI, sorted() returns an iterator, not a list.
from piff.
I don't think so...
>>> type(sorted(glob.glob('*.fits')))
<type 'list'>
or in Python 3.4 (where I thought maybe they changed the nature of this function)
>>> type(sorted(glob.glob('*.fits')))
<class 'list'>
from piff.
I did not know about the sorted() thingy. I always did it the way Erin did.
In any event, as Mike has said, the original point still stands that we
need to guarantee 1-1 matching. It may be that simply using sorted() does
the trick if you have filenames that are all the same except for some
obviously sortable index, which is the "expected" behavior.
On Wed, Jun 22, 2016 at 10:25 AM, Erin Sheldon [email protected]
wrote:
I was clearly confused
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
#26 (comment), or mute
the thread
https://github.com/notifications/unsubscribe/AE7bEx0CKEX-E3kLVWiCbgzI_QV6j8SZks5qOW_0gaJpZM4I78t1
.
from piff.
In my experience it is best to determine such things algorithmically. For example, if I'm processing an exposure I can generate the file names for the ccds in the exposure using a simple algorithm based on the DESDM run identifier. The base name is known and the suffix is based on ccd number.
I recommend doing that by default. e.g. specify an exposure you want to process by name or id, and the code looks for it in the usual place ($DESDATA/OPS/red/...)
and does its thing.
This has the advantage that if a file is missing you will notice it; you generated the file list and you know what should be there, if the file is missing the code will crash. Also there are usually auxilliary files you will need to use that will be generated using the same algorithms, and don't need to be determined from the input file list.
Then if you want to be able to do something different, perhaps for testing, that can be an alternative method, e.g. specifying an exact list of files, or a pattern as in the above config.
from piff.
This is now fixed on branch #20. There are a few ways allowed to specify the file lists. From the doc string:
There are a number of ways to specify the input files (parameters `images` and `cats`):
1. If you only have a single image/catalog, you may just give the file name directly
as a single string.
2. For multiple images, you may specify a list of strings listing all the file names.
3. You may specify a string with ``{chipnum}`` which will be filled in by the chipnum
values given in the `chipnums` parameter using ``s.format(chipnum=chipnum)``.
4. You may specify a string with ``%s`` (or perhaps ``%02d``, etc.) which will be filled
in by the chipnum values given in the `chipnums` parameter using ``s % chipnum``.
5. You may specify a string that ``glob.glob(s)`` will understand and convert into a
list of file names. Caveat: ``glob`` returns the files in native directory order
(cf. ``ls -f``). This can thus be different for the images and catalogs if they
were written to disk out of order. Therefore, we sort the list returned by
``glob.glob(s)``. Typically, this will result in the image file names and catalog
file names matching up correctly, but it is the users responsibility to ensure
that this is the case.
The `chipnums` parameter specifies chip "numbers" which are really just any identifying
number or string that is different for each chip in the exposure. Typically, these are
numbers, but they don't have to be if you have some other way of identifying the chips.
There are a number of ways that the chipnums may be specified:
1. A single number or string.
2. A list of numbers or strings.
3. A string that can be ``eval``ed to yield the appropriate list. e.g.
`[ c for c in range(1,63) if c is not 61 ]`
4. None, in which case range(len(images)) will be used. In this case options 3,4 above
for the images and cats parameters are not allowed.
from piff.
Related Issues (20)
- Reading WCS solution of ZTF images HOT 1
- Stars Center on the stamps HOT 7
- psf.draw | wcs is not a list for single chip fit HOT 5
- Add equality testing HOT 1
- very large PSF (after re-installation) | Bad fit HOT 6
- regularize in a hacky way HOT 3
- Examples in documentation & `examples/` fail for v1.2 HOT 3
- Piff catastrophically fails when PixelGrid size is larger than input star images HOT 3
- Inconsistent piff log message levels HOT 2
- Fix max_snr weight adjustment.
- Make it easy to run just the star selection without fitting a PSF model
- incorrect error message with missing keyword
- coadd_object_id in psf_stars extension is stored as an f8 instead of an i8 HOT 3
- Add optional high order moments to the HSM output file. HOT 4
- Let weight, badpix be different files from image.
- Strange PSF fitting HOT 5
- Request: Release new version
- test suite needs pytest<8 and galsim<2.5 HOT 1
- Add option to outlier-reject reserve stars at the end of the processing HOT 1
- Serialization for embedding in larger (Rubin) files HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from piff.