sherpa / sherpa-test-data Goto Github PK

Test data and scripts for Sherpa

Python 100.00%

sherpa-test-data's Introduction

Table of Contents

Sherpa
License
How To Install Sherpa
History
- Release History

Sherpa

Sherpa is a modeling and fitting application for Python. It contains a powerful language for combining simple models into complex expressions that can be fit to the data using a variety of statistics and optimization methods. It is easily extensible to include user models, statistics, and optimization methods. It provides a high-level User Interface for interactive data-analysis work, such as within a Jupyter notebook, and it can also be used as a library component, providing fitting and modeling capabilities to an application.

What can you do with Sherpa?

fit 1D (multiple) data including: spectra, surface brightness profiles, light curves, general ASCII arrays
fit 2D images/surfaces in Poisson/Gaussian regime
build complex model expressions
import and use your own models
use appropriate statistics for modeling Poisson or Gaussian data
import new statistics, with priors if required by analysis
visualize the parameter space with simulations or using 1D/2D cuts of the parameter space
calculate confidence levels on the best fit model parameters
choose a robust optimization method for the fit: Levenberg-Marquardt, Nelder-Mead Simplex or Monte Carlo/Differential Evolution.

Documentation for Sherpa is available at Read The Docs and also for Sherpa in CIAO.

A Quick Start Tutorial is included in the notebooks folder and can be opened with an ipython notebook.

License

This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version. A copy of the GNU General Public License can be found in the LICENSE file provided with the source code, or from the Free Software Foundation.

How To Install Sherpa

Full installation instructions are part of the Read The Docs documentation, and should be read if the following is not sufficient.

It is strongly recommended that some form of virtual environment is used with Sherpa.

Sherpa is tested against Python versions 3.9, 3.10, and 3.11.

The last version of Sherpa which supported Python 2.7 is Sherpa 4.11.1.

Using Conda

Sherpa is provided for both Linux and macOS operating systems running Python 3.9, 3.10, and 3.11. It can be installed with the conda package manager by saying

$ conda install -c https://cxc.cfa.harvard.edu/conda/sherpa -c conda-forge sherpa

Using pip

Sherpa is also available on PyPI and so can be installed with the following command (which requires that the NumPy package is already installed).

% pip install sherpa

Building from source

Source installation is available for platforms incompatible with the binary builds, or for when the default build options are not sufficient (such as including support for the XSPEC model library). The steps are described in the building from source documentation.

History

Sherpa is developed by the Chandra X-ray Observatory to provide fitting and modelling capabilities to the CIAO analysis package. It has been released onto GitHub for users to extend (whether to other areas of Astronomy or in other domains).

Release History

4.16.1: 21 May 2024

4.16.0: 17 October 2023

4.15.1: 18 May 2023

4.15.0: 11 October 2022

4.14.1: 20 May 2022

4.14.0: 07 October 2021

4.13.1: 18 May 2021

4.13.0: 08 January 2021

4.12.2: 27 October 2020

4.12.1: 14 July 2020

4.12.0: 30 January 2020

4.11.1: 1 August 2019

4.11.0: 20 February 2019

4.10.2: 14 December 2018

4.10.1: 16 October 2018

4.10.0: 11 May 2018

4.9.1: 01 August 2017

4.9.0: 27 January 2017

4.8.2: 23 September 2016

4.8.1: 15 April 2016

4.8.0: 27 January 2016

sherpa-test-data's People

Contributors

Stargazers

Watchers

Forkers

dougburke jbudynk hamogu

sherpa-test-data's Issues

Rename 'ciao4.3' directory to 'ciao'

To go along with the new test infrastructure, we may want to rename the "ciao4.3" directory to simply "ciao." This involves changing the datadir pointers in the sherpa source code.

Refractor files under load_template* directories

There are multiple copies of the same files in directories

ciao4.3/load_template_interpolator/
ciao4.3/load_template_with_interpolation/
ciao4.3/load_template_without_interpolation/

The file names are named bb_*.dat.

This issue is minor. Refractoring the files with symlinks has little to no impact on the size of the repository since the original files are still in the git history. However, being consistent in organization is a plus.

Update ciao4.3/template_restore tests

In the old testdata directory (/data/scialg/testdata/ciao4.3/template_restore/), there were ~20 large files and a commented-out test.py.

We need to update ciao4.3/template_restore/test.py so that it actually runs tests. The test is currently commented out.

Review removed threads

In the old testdata directory, we found that many of the threads/ directories aren't tested by the sherpa regression tests. We removed these before creating sherpa-test-data. We should

figure out why they aren't tested
create regression tests for these threads if necessary, and
see if there's an external regression test suite that uses these threads

For a list of the removed threads and files, see section "Files" under 4.8->features->new-test-infrastructure->testdata on the Sherpa Dokuwiki.

Flatten test data directory

The test data directory structure is more complicated than necessary. Flatten out the files and store everything in the base directory. The symlinks may not carry over correctly when decompressing the tar file, and may instead be copied as duplicate files, inflating the directory size.

Each thread directory under sherpatest/ciao4.3 has a *.py script that runs the regression test for that science case. These files are located and run by calling sherpa.astro.tests.test_astro::run_threads() (at sherpa/sherpa@be36cf3c9). We need to make sure the tests run the proper script.

This may require changes in Sherpa.

Should be store scripts / notes used to generate the test files?

Some of the test files in the repro are probably hand-written and easy to understand. Others, especially those that are real Chandra/XMM/... data or are clearly made with CIAO/SAS/FTOOLS/... . I think it would be good to also store the scripts to make those files. This is useful for two cases:

If certain details in the data format change (e.g. the header keywords) we may want to regenerate those tests file to make sure we test against the current form (Unlikely to have much impact for OGIP compliant files, but we did ass new header keywords in repro 5 for Chandra, so it's not impossible that that happen again and might conceivably impact e.g. image tests.)
It makes it easier to add files to the existing collection without replacing anything. For example, I need a background ARF and RMF for a spectrum as well as the full set of source file. We have the "7994." files in the repro, but I would really like ot run a test with a background arf that's different from the source arf. If I had the script, I might just re-run it, get a the new file I want and add it to the collection without touching the existing "7994." files. (Since git is not good with binary files, it's good not to replace them if we don't have to.) In this particular case, I can probably work around it, but it got me thinking - thus this issue.

I would not retroactively make up things (it's not worth that much work), but simply encourage people to leave notes or scripts in a /srcipt sub-directory if adding files in the future.