Code Monkey home page Code Monkey logo

theliwrapper's Introduction

# TheliWrapper

The TheliWrapper is a project to simplify the process of astronomical image reduction with the THELI (software package for astronomical image reduction in optical, near- and mid-infrared) significantly.
It combines the simplicity of the THELI GUI's preconfigured instruments and parameter files with the ability to do large scale batch processing for large observation projects.

For an overview of the TheliWrapper's features, we can refer to the THELI GUI webpage:

THELI is a powerful and easy-to-use package for astronomical image reduction, offering e.g.

  • Automated reduction of optical, NIR and MIR data
  • 90 pre-configured instruments
  • Parallelisation
  • Flexible background correction
  • Automatic distortion correction and creation of large mosaics
  • Highly flexible coaddition: e.g. locking onto proper motion targets, various projections, etc
  • Optimised weighting schemes and defect detection
  • Crosstalk and non-linearity correction
  • Atmospheric transparency correction
  • Absolute flux calibration
  • Extensive documentation and online help

About this README

This file will guide you through the installation of the program and inform you about the current development stage or known issues. Any other topic will be covered by the the project's Wiki-page, giving more documentation and a step-by-step usage demonstration you can download and try yourself.
For any further problems, please contact the project maintainer(s) (see below).

Requirements

The TheliWrapper is entirely written in Python 3, but it depends on external components which are written in Python 2 or C/C++. Here is a list of components most users will need to install to use the wrapper.

Python:

  • Python 2 and 3 (version >= 2.5 and version >= 3.4)
  • Python 2 packages: numpy, scipy, matplotlib, pyfits
  • Python 3 packages: astropy or pyfits (optional, but recommended)

C-libraries:

  • Python C-headers
  • FFTW, GSL (GNU Scientific Library)
  • cfitsio, CCfits
  • LibTIFF, LibPNG (depends on your system)

Programs:

  • csh, wget
  • ImageMagick
  • THELI package, requires CDS client
  • scripts and configuration files from THELI graphical user interface (GUI)
  • scamp (optional but recommended), requires PLplot with cairo driver

Installation

These instructions should guide you through the steps of installing the TheliWrapper together with its dependencies:

CDS client

THELI depends on the CDS client. Download the most recent version, extract it and install it with

./configure
make
make install

THELI

To install THELI, download and extract it and run the install scipt in pipesetup/ with install.sh -m "ALL" to build the package from source.

Scamp

Even though THELI offers alternatives, it is highly recommended to install scamp to compute precise astrometric solutions. It is not part of the package itself, so has to be installed separately and requires PLplot with the cairo-driver. On some linux distributions (e.g. Ubuntu, Linux Mint or Arch Linux) scamp can be found precompiled in the package repositories.
Otherwise download it and try to build it from source (which can be tricky) with

./configure
make
make install

THELI GUI

Since version 0.6.1 of the TheliWrapper
If you want to avoid installing Qt3 you can use a modified version which ships all functionality of the GUI except the graphical window (which requires Qt3).

Download and install the THELI GUI by using ./install.sh and copy the scamp binary to the binary folder of your THELI installation:

 [path to THELI]/bin/[your platform]/

The GUI requires Qt3 which has to be installed manualy on most systems.

TheliWrapper

To install the TheliWrapper, just download the latest release and copy and extract it to your preferred destination. When you run theli.py for the first time it may ask you to locate the THELI installation folder (root folder which contains the theli link) and will set up itself automatically.
Whenever you change your THELI installation, it will be necessary to delete /home/janluca/.theli/theli_paths.py such that it will be recreated.

Project progress and known issues

This project is under development, even though the most important data recution steps are implented. Yet missing are:

  • Photometry (direct and indirect)
  • Constant sky model subtraction
  • Copying and restoring the astrometric solution to the image header manually

If you work with (near/mid) infrared data the following tools are availble but still experimental:

  • Cross talk correction
  • Squence splitting
  • Chop/nod sky subtraction
  • Collapse correction

If you have any problems or notice any unexpected behaviour, please contact the maintainer(s) or create a new issue on the project's GitHub-page.

Maintainers

Jan Luca van den Busch (Argelander Institute for Astronomy)

Acknowledgements

Please include the URL of this repository in your acknowledgments and cite the following two papers when publishing your scientific work based on THELI:

Schirmer M. 2013, ApJS, 209, 21: THELI GUI โ€“ Convenient reduction of optical, near- and mid-infrared imaging data

Erben, T., Schirmer, M., Dietrich, J. et al. 2005, AN, 326, 432: GaBoDS: The Garching-Bonn Deep Survey. IV. Methods for the image reduction of multi-chip cameras demonstrated on data from the ESO Wide-Field Imager

theliwrapper's People

Contributors

jlvdb avatar

Watchers

 avatar  avatar

theliwrapper's Issues

change list_filters implementation

list_filters does not work, if the instrument is not implemented explicitly. This will cause Reduction.init to terminate stating taht no valid FITS files are in the current folder

class Folder _index updating

##################  SCIENCE  ####################################################
Session info:       version 0.5.3, 4/4 CPU(s), 23 FRAMES
Instrument:         ACAM@WHT
Main folder:        /.../NewFolder
Bias folder:        BIAS
Flat folder:        FLAT
Science folder:     SCIENCE

> Splitting FITS, correcting headers (bias)
> Splitting FITS, correcting headers (flat)
> Splitting FITS, correcting headers (science)

> Processsing BIASes

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/astropy/io/fits/file.py", line 118, in __init__
    self.name, _ = urllib.request.urlretrieve(fileobj)
  File "/usr/lib/python3.4/urllib/request.py", line 186, in urlretrieve
    with contextlib.closing(urlopen(url, data)) as fp:
  File "/usr/lib/python3.4/urllib/request.py", line 161, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib/python3.4/urllib/request.py", line 448, in open
    req = Request(fullurl, data)
  File "/usr/lib/python3.4/urllib/request.py", line 266, in __init__
    self.full_url = url
  File "/usr/lib/python3.4/urllib/request.py", line 292, in full_url
    self._parse()
  File "/usr/lib/python3.4/urllib/request.py", line 321, in _parse
    raise ValueError("unknown url type: %r" % self.full_url)
ValueError: unknown url type: '/.../NewFolder/FLAT/r2293572.fit'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/.../bin/TheliWrapper/theli.py", line 20, in <module>
    getattr(project, job["func"])(*jobargs)
  File "/.../bin/TheliWrapper/system/reduction.py", line 524, in process_flats
    self.check_filters()
  File "/.../bin/TheliWrapper/system/reduction.py", line 201, in check_filters
    new = dfolder.filters()
  File "/.../bin/TheliWrapper/system/folder.py", line 166, in filters
    os.path.join(self.abs, f), ["FILTER"])[0])
  File "/.../bin/TheliWrapper/system/base.py", line 160, in get_FITS_header_values
    with pyfits.open(file) as fits:
  File "/usr/lib/python3/dist-packages/astropy/io/fits/hdu/hdulist.py", line 116, in fitsopen
    return HDUList.fromfile(name, mode, memmap, save_backup, **kwargs)
  File "/usr/lib/python3/dist-packages/astropy/io/fits/hdu/hdulist.py", line 248, in fromfile
    save_backup=save_backup, **kwargs)
  File "/usr/lib/python3/dist-packages/astropy/io/fits/hdu/hdulist.py", line 759, in _readfrom
    ffo = _File(fileobj, mode=mode, memmap=memmap)
  File "/usr/lib/python3/dist-packages/astropy/io/fits/file.py", line 126, in __init__
    raise IOError('File does not exist: %r' % fileobj)
OSError: File does not exist: '/.../NewFolder/FLAT/r2293572.fit'

jobs

Give name from GUI check box as headline in help

TODO

-- implement photometry
-- header update/restore
-- const. sky subtraction helper
-- xtalk: verify ButtonIDs, better help, ...
-- revise help strings
-- identify unmatched parameters
-- extensive parameter testing
-- test MIR/NIR data, sequence handling, link creation/resolving

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.