Code Monkey home page Code Monkey logo

simmer's Introduction

SImMER

arXiv Tests codecov Maintainability License: MIT Build Status PyPI version Conda Version Code style: black Yaml Lint

Repository for developing the SImMER image reduction pipeline. If you'd like to help out, take a look at our ways to contribute. If you use this package in your research, please cite our paper.

Installation

To install with conda (the recommended method), run

conda config --add channels conda-forge
conda install simmer

To install with mamba, run

mamba install simmer

To install with pip, run

pip install simmer

Or, to install from source, run

python3 -m pip install -U pip
python3 -m pip install -U setuptools setuptools_scm pep517
git clone https://github.com/arjunsavel/SImMER.git


cd simmer
python3 -m pip install -e .

Documentation

To get started, read the docs at our readthedocs site.

simmer's People

Contributors

arjunsavel avatar cdressing avatar dependabot[bot] avatar github-actions[bot] avatar holdengill avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

simmer's Issues

Config File method column syntax

The config files require each target have 'saturated' in the 'Method' column. Cannot be capitalized (i.e. 'Saturated' does not work). The error received if you put a capitalized 'S' is screenshotted below, and it happens during registration:

image

Generalize config-handling

Is your feature request related to a problem? Please describe.

If a star is observed twice with varying integration times, it is not separated into different folders. Additionally, if it is observed with a gap in the middle, the current range format would not be able to handle this.

Describe the solution you'd like

Generalize the handling of file numbers.

Additional context

Better handle warnings

Describe the bug

The warnings are just thrown away; they should be better incorporated. Perhaps into a logger.

Better automate versioning

Is your feature request related to a problem? Please describe.

Should not need to go into multiple sources to update the version — this is ripe for error!

Describe the solution you'd like

Ideally, working through the GitHub release, which gets propagated to the Python init, or something to that effect...

Additional context

Refactor registration

Is your feature request related to a problem? Please describe.

Codeclimate has a lot of smells for this file.

Describe the solution you'd like

A few days of refactoring is estimated to be necessary to get rid of duplicated code, etc.

Describe alternatives you've considered

Additional context

Image registration reads from folders, not config

Describe the bug

When determining what to perform registration on, SImMER checks folder-by-folder; this can lead to errors.

To Reproduce

Create a random folder in the reduced directory.

Expected behavior

The config file should be read through, and the correct filters from the config should be referenced.

Screenshots

Screen Shot 2020-07-20 at 3 11 48 PM

Additional context

N.A

Pin or fix pandas for read_excel

Describe the bug

There's an issue in read_excel past the pandas 1.0 update. The dependency should either be pinned lower or the bug should be fixed.

To Reproduce

Run the tests or run create_config on an Excel file.

Screenshots

Screen Shot 2020-08-10 at 5 40 09 PM

Additional context

Other dependencies should also probably be pinned.

Descriptive errors for file-matching

Is your feature request related to a problem? Please describe.

When darks, flats, or skies aren't matched correctly, a FileNotFound error is thrown. This doesn't say much about what's actually going on.

Describe the solution you'd like

It'd be preferable to have an error thrown that better sums up the problem, e.g. "Missing flat corresponding to..."

Screen shots
Screen Shot 2020-07-16 at 6 31 44 PM

Additional context

N/A

Skies-Labelled Stars

When a star has an additional label 'skies' after the ID number, it screws up the final output image/registration. The 'skies' term must be removed from the 'object' column, and a new column labelled 'comments' must be added, where you put 'sky' for the previously labeled 'skies' objects.

Then when instantiating, need to use 'take_skies=True'. This currently results in the registration not identifying any objects, and the all_driver code stops once it reaches the skies portion.

Logging

Is your feature request related to a problem? Please describe.

Print statements aren't the best way to go about communicating to users.

Describe the solution you'd like

A mixture of tqdm and the Python logging module would likely serve this need.

Additional context

Simplify dependencies

Is your feature request related to a problem? Please describe.

There are a few dependencies that overlap, e.g. xlrd and openpyxl.

Describe the solution you'd like

Find the overlaps and remove them!

Additional context

Limiting the number of dependencies enables an easier conda environment solve + (IMO) makes the whole new user interface easier.

Check for saturated and wide binary cases

A user might not realize that their image is saturated or that their binaries are too similar in magnitude, so having the pipeline check for these cases and alert the user would be helpful.

Run notebooks in documentation build

Is your feature request related to a problem? Please describe.

The docs will sometimes include notebooks that don't represent the latest code.

Describe the solution you'd like

Each notebook should be run on the latest SImMER version before being published to account for any errors.

Additional context

This will definitely increase the doc deploy runtime (up to 25 min, probably), but I think it's worth the wait.

Move away from Amazon S3...

Is your feature request related to a problem? Please describe.

...because they charge data transfer fees.

Describe the solution you'd like

Dropbox, perhaps?

Additional context

Remove aodirs references

Describe the bug

Aodirs is no longer used; therefore, it should be removed from documentation.

To Reproduce

Use one of the example notebooks.

Expected behavior

The notebooks should run!

Screenshots

Additional context

User input type-checking

Is your feature request related to a problem? Please describe.

No problems...yet!

Describe the solution you'd like

Ideally, users should be let know if they try to pass something incorrect to the code. That could be anything from a damaged FITS file to a .pkl file.

Additional context

N/A

Add developers section to documentation

Is your feature request related to a problem? Please describe.

There are a number of features/services that are being used. It'd make sense to document this clearly.

Describe the solution you'd like

Just adding another page to the readthedocs.

Additional context

Nope!

Update quickstarts

Is your feature request related to a problem? Please describe.

The Shane quickstart has been updated, but PHARO hasn't. Additionally, the Shane quickstart should be simplified a bit.

Describe the solution you'd like

Updated config reading and writing + include 4-quadrant search.

Additional context

Make it clear that examples aren't complete

Describe the bug
The example notebooks are just skeletons.

To Reproduce
The cells for shane_quickstart don't run.

Expected behavior
They should run!
Screenshots
N/A

Additional context
None!

Add_dark_exposure for CSV logsheets

Is your feature request related to a problem? Please describe.

Add_dark_exposures currently only works when logsheets are Excel files.

Describe the solution you'd like

This should be extended to include CSVs, using pandas.

Additional context

Speed up saturated registration mode

Is your feature request related to a problem? Please describe.
The saturated image registration mode is currently orders of magnitude slower than the default mode. If it were faster, though, its level of robustness would make it the clearly preferred registration method.

Describe the solution you'd like
Using Cython or Numba would probably be best.

Describe alternatives you've considered
Parallelizing the code might be helpful, but not sure if this is too dependent on the number of cores available to the user.

Resource Warning

Describe the bug

When tests are run, occasionally the below error is thrown.

To Reproduce

Run tests a number of times.

Expected behavior

Tests should pass.

Screenshots

Screen Shot 2020-06-03 at 5 57 12 PM

Additional context

Charles reported a similar error.

Test search_headers

Is your feature request related to a problem? Please describe.

The search_headers function is currently only tested on files that don't have the stated issue.

Describe the solution you'd like

Designated testing on the file that caused the issue.

Additional context

GUI for SImMER

Is your feature request related to a problem? Please describe.
Setting up a config, checking file paths, and running code can take a bit of time.

Describe the solution you'd like
A graphical user interface would allow users to drag and drop their data into the pipeline with ease.

Additional context
N/A

Add plotting kwargs

Is your feature request related to a problem? Please describe.
The image as produced by the pipeline is not customizable. Producing a system to pass plotting kwargs to matplotlib would thus be beneficial.

Describe the solution you'd like
A simple **kwargs arg in the main body could suffice.

Describe alternatives you've considered

  • A plotting object that is optionally created by the user and referred to
  • A new file altogether

Compute contrast curves

Is your feature request related to a problem? Please describe.
In many imaging campaigns, scientists are interested in quantifying the non-detection of companions to a bright source.

Describe the solution you'd like
Computing contrast curves is the canonical method by which to quantify this non-detection.

Make wide binary search work in Jupyter environment

Describe the bug

Based on the way that the plots are produced, the user input isn't asked for in a Jupyter environment, causing the program to terminate incorrectly.

To Reproduce

Attempt a reduction with "wide" method in a Jupyter environment.

Expected behavior

The

Screenshots

Additional context

Bump pandas to 1.1

Is your feature request related to a problem? Please describe.

It's best to be on the latest versions of packages.

Describe the solution you'd like

Pin pandas higher, and ensure that the parse-cols is taken out of the read_excel command.

Additional context

Create logo

A logo would be a nice to put in this readme, on readthedocs, etc.

Create "Method" column

Is your feature request related to a problem? Please describe.

When creating a config file, the "Method" column usually goes unnoticed until everything breaks.

Describe the solution you'd like

Actually create the "Method" column if one isn't specified.

Additional context

"Remembering" the previous selection of a wide binary

This feature would limit manual user input by finding the two stars in the image, determining their relative positions, and using the information from the first user click to determine which of the two stars is the primary.

Put on PyPI

Is your feature request related to a problem? Please describe.

It's easiest for people to download packages if they're pip-installable.

Describe the solution you'd like

Register the package on PyPI.

Additional context

Make sure that the publishing only happens if this isn't a beta version!

Depfu Error: No dependency files found

Hello,

We've tried to activate or update your repository on Depfu and couldn't find any supported dependency files. If we were to guess, we would say that this is not actually a project Depfu supports and has probably been activated by error.

Monorepos

Please note that Depfu currently only searches for your dependency files in the root folder. We do support monorepos and non-root files, but don't auto-detect them. If that's the case with this repo, please send us a quick email with the folder you want Depfu to work on and we'll set it up right away!

How to deactivate the project

  • Go to the Settings page of either your own account or the organization you've used
  • Go to "Installed Integrations"
  • Click the "Configure" button on the Depfu integration
  • Remove this repo (arjunsavel/SImMER) from the list of accessible repos.

Please note that using the "All Repositories" setting doesn't make a lot of sense with Depfu.

If you think that this is a mistake

Please let us know by sending an email to [email protected].


This is an automated issue by Depfu. You're getting it because someone configured Depfu to automatically update dependencies on this project.

Extend to other instruments

This code was made modular for a reason — we'd like to be able to use it on a variety of telescope/camera combinations! If you'd like to extend to another facility, please shoot Arjun ([email protected]) an email.

Document image_drivers

Is your feature request related to a problem? Please describe.

image_driver and config_driver aren't well-represented in the documentation.

Describe the solution you'd like

There should be notebooks attached to this.

Additional context

Refactor test suite

Is your feature request related to a problem? Please describe.

The test suite is looking a bit messy — there's a good bit of replicated code.

Describe the solution you'd like

Better usages of classes could solve this, with simple method calls.

Additional context

N/A

Ensure 4-quadrant PHARO functionality

Is your feature request related to a problem? Please describe.
Currently, this pipeline does not actually take in raw PHARO data.

Describe the solution you'd like
Being able to take 4-quadrant data as input would greatly increase the usefulness of this code. The implementation currently exists, but it must be documented and tested.

Test wide binary case

Our current wide binary scenario involves user input, so it doesn't fit neatly into our existing test implementations.

Test/document config file better

Is your feature request related to a problem? Please describe.

The config file preparation is the most user-intensive aspect of the reduction process, so it should be more streamlined.

Describe the solution you'd like

A rigorous test suite and a tutorial

Additional context

ASAP!

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.