Cartesian representation is smaller and allows for more efficient computation of back projection from stacked calibrated event list. Also in this representation the process is a bit easier to follow.

This file comes from old template and shouldn't be here anymore.

There is an issue with the naming convention of the chianti7p1_rad_loss.txt data file on hesperia which we were discussing a few weeks ago. As you can see, the chianti version is hard-coded into the file name. This is an issue when it comes to writing sunpy functions which use this file as it would require the file name to be changed manually every time there is an update. I propose we try to get this naming convention changed to simply chianti_rad_loss.csv. Changing it to a csv file makes it more suitable to manipulation by python’s csv module. Also, the version used is still contained within the comments in the first few lines of the file. I also propose that we ensure that all comment lines at the top of the file are prefixed by a certain character, either, #, ; or whatever. This makes it easier to parse.

Describe the bug

goes = fits.open('OR_SUVI-L1b-Fe171_G17_s20221391205103_e20221391205113_c20221391205274.fits.gz')
print(goes.info())
sunkit_instruments.suvi.files_to_map(goes)

Filename: OR_SUVI-L1b-Fe171_G17_s20221391205103_e20221391205113_c20221391205274.fits.gz
No.    Name      Ver    Type      Cards   Dimensions   Format
  0  DATA          1 PrimaryHDU     131   (1280, 1280)   int16 (rescales to float32)   
  1  DQF           1 ImageHDU        18   (1280, 1280)   uint8   


None
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Input In [35], in <cell line: 3>()
      1 goes = fits.open('OR_SUVI-L1b-Fe171_G17_s20221391205103_e20221391205113_c20221391205274.fits.gz')
      2 print(goes.info())
----> 3 sunkit_instruments.suvi.files_to_map(goes)

AttributeError: module 'sunkit_instruments' has no attribute 'suvi'

What happened?

I want to open a FITS file of GOES-SUVI data as a sunpy map using the sunkit_instruments.suvi , but sunkit_instruments has no suvi attribute, although everything is up to date.

System Details

• sunkit_instruments Version: 0.3.0
• sunpy Version: 4.0.1
• python Version: 3.9

Installation method

pip, conda

Reimplement the IRISMapCube.convert_to() and IRISMapCubeSequence.convert_to() methods as a single function that takes a sunraster cube or sequence and converts the data between counts/photons/radiance units.

• This is probably best implemented with two hidden functions. The first to act on cubes, the second on sequences by calling the cube conversion function for each cube in the sequence, and the user facing function can just be a wrapper that checks the type of the input and applies the relevant function.
• There should be some functions already in iris_tools/utils/iris_utils that may require only a little modification to be helpful in this.

Open PR to iris branch.

Describe the bug

All in the title; see https://docs.sunpy.org/projects/sunkit-instruments/en/stable/api/sunkit_instruments.lyra.get_lytaf_events.html#sunkit_instruments.lyra.get_lytaf_events

To Reproduce

import sunpy

sunpy.map.Map(...)
etc

Screenshots

No response

System Details

No response

Installation method

No response

Describe the feature

the data from the Skylab ATM is available from NRL, and it would be really nice to have some functionality to search for and load the data (especially the overlappograms and spectra)

Data is available here:

https://lasco-www.nrl.navy.mil/skylab/spectral/skylab/S082A/
https://lasco-www.nrl.navy.mil/skylab/spectral/skylab/S082B/

and some documentation here:
https://lasco-www.nrl.navy.mil/skylab/spectral/skylab/docs/pdf/

get_goes_event_list queries the HEK. @Cadair says that the HEK doesn't seem to do well with TimeRange values measured in large numbers of months. I've presently been waiting 16 hours for the event list return for a 4 year time range for flares > C1.0 (I assume it is actually hung). The equivalent method (rd_gev.pro) in IDL solarsoft returns in a few seconds for the same inputs. That implementation predates the HEK's existence by a couple decades. It requires that your local solarsoft database includes the GOES events files, so to stay up to date, your system would need to be running sswdb update periodically.

But there is another way. NOAA posts the event list directly on their site. I note that at the moment, it hasn't been updated in 6 months though. I'm asking around to see if there's another site. Scraping this site, of course, would not have a guarantee of long-term reliability as evidenced by the fact that my bookmark to the page was out of date just now and it did not redirect me to the link above. Still, scraping should be much quicker than going through the HEK. So this new feature could be an alternative implementation that triggers on, e.g., a timeout, try/catch, or by user specified kwarg.

Implementation parsing note:
The format that NOAA uses for the date in those files is very wonky. An example: 31777170628, which seems to be
[31777] = some sort of static identifier
[170628] = 2017-06-28

Description

The goes_xrs.calculate_temperature_em function does not work for the new netCDF GOES files.

Steps to Reproduce

from sunkit_instrument.goes import calculate_temperature_em
from sunpy.time import parse_time
from sunpy import timeseries
from sunpy.net import Fido, attrs as a

result = Fido.search(
    a.Time('2020/11/29 00:00', '2020/11/29 23:59:59.999'),
    a.Instrument.xrs,
    a.goes.SatelliteNumber(16)
)
f = Fido.fetch(result)
ts = timeseries.TimeSeries(f)
em = calculate_temperature_em(ts)

raises the following error,

---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
<ipython-input-27-23fdf0b01a7e> in <module>
----> 1 em = calculate_temperature_em(ts)

~/miniconda3/envs/sunpy-user/lib/python3.8/site-packages/sunpy/extern/sunkit_instruments/goes_xrs/goes_xrs.py in calculate_temperature_em(goests, abundances, download, download_dir)
    282         goests.quantity("xrsb"),
    283         goests.quantity("xrsa"),
--> 284         satellite=goests.meta.metas[0]["TELESCOP"].split()[1],
    285         date=goests.to_dataframe().index[0],
    286         abundances=abundances,

~/miniconda3/envs/sunpy-user/lib/python3.8/site-packages/sunpy/util/metadata.py in __getitem__(self, key)
    157         Override ``[]`` indexing.
    158         """
--> 159         return OrderedDict.__getitem__(self, key.lower())
    160
    161     def __setitem__(self, key, value):

KeyError: 'telescop'

There does not seem to be any info about the satellite number in the headers of these files. Correcting metadata manually,

ts.meta.metas[0]['TELESCOP'] = 'GOES 16'

gives another exception

---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
<ipython-input-43-23fdf0b01a7e> in <module>
----> 1 em = calculate_temperature_em(ts)

~/miniconda3/envs/sunpy-user/lib/python3.8/site-packages/sunpy/extern/sunkit_instruments/goes_xrs/goes_xrs.py in calculate_temperature_em(goests, abundances, download, download_dir)
    279
    280     # Find temperature and emission measure with _goes_chianti_tem
--> 281     temp, em = _goes_chianti_tem(
    282         goests.quantity("xrsb"),
    283         goests.quantity("xrsa"),

~/miniconda3/envs/sunpy-user/lib/python3.8/site-packages/astropy/units/decorators.py in wrapper(*func_args, **func_kwargs)
    249             # Call the original function with any equivalencies in force.
    250             with add_enabled_equivalencies(self.equivalencies):
--> 251                 return_ = wrapped_function(*func_args, **func_kwargs)
    252
    253             valid_empty = (inspect.Signature.empty, None)

~/miniconda3/envs/sunpy-user/lib/python3.8/site-packages/sunpy/extern/sunkit_instruments/goes_xrs/goes_xrs.py in _goes_chianti_tem(longflux, shortflux, satellite, date, abundances, download, download_dir)
    444
    445     # FIND TEMPERATURE AND EMISSION MEASURE FROM FUNCTIONS BELOW
--> 446     temp = _goes_get_chianti_temp(
    447         fluxratio,
    448         satellite=satellite,

~/miniconda3/envs/sunpy-user/lib/python3.8/site-packages/sunpy/data/data_manager/manager.py in wrapper(*args, **kwargs)
    111                     self._file_cache[name] = {}
    112                 self._file_cache[name][self._namespace] = file_path
--> 113                 result = func(*args, **kwargs)
    114                 self._namespace = None
    115                 return result

~/miniconda3/envs/sunpy-user/lib/python3.8/site-packages/sunpy/data/data_manager/manager.py in wrapper(*args, **kwargs)
    111                     self._file_cache[name] = {}
    112                 self._file_cache[name][self._namespace] = file_path
--> 113                 result = func(*args, **kwargs)
    114                 self._namespace = None
    115                 return result

~/miniconda3/envs/sunpy-user/lib/python3.8/site-packages/astropy/units/decorators.py in wrapper(*func_args, **func_kwargs)
    249             # Call the original function with any equivalencies in force.
    250             with add_enabled_equivalencies(self.equivalencies):
--> 251                 return_ = wrapped_function(*func_args, **func_kwargs)
    252
    253             valid_empty = (inspect.Signature.empty, None)

~/miniconda3/envs/sunpy-user/lib/python3.8/site-packages/sunpy/extern/sunkit_instruments/goes_xrs/goes_xrs.py in _goes_get_chianti_temp(fluxratio, satellite, abundances, download, download_dir)
    596         for row in csvreader:
    597             modeltemp.append(float(row["log10temp_MK"]))
--> 598             modelratio.append(float(row[label]))
    599     modeltemp = np.asarray(modeltemp)
    600     modelratio = np.asarray(modelratio)

KeyError: 'ratioGOES16'

I'm still trying to track down what exactly the root cause of this is, but it seems like the requested CHIANTI emission tables do not include the key corresponding to GOES 16.

System Details

• SunPy Version: 3.0
• Astropy Version: 4.2
• Python Version: 3.8
• OS information: macOS

Provide a general description of the issue or problem.

The example at https://docs.sunpy.org/projects/sunkit-instruments/en/stable/generated/gallery/plot_suvi_thematic_map.html#sphx-glr-generated-gallery-plot-suvi-thematic-map-py currently uses a custom _download_file function. To integrate better with the sunpy ecosystem this should be changed to using

• parfive (https://github.com/cadair/parfive) to download the data.
• or ideally searching for and downloading using sunpy.net.Fido.

Reimplement calculate_orbital_wavelength_variation as a function that takes sunraster Spectrogram/Raster objects.

• See the correct_orbital_wavelength_variation method on the IRISSpectrogramCube class on the IRISpy branch of sunraster for ideas.
• Most of the work is done by the _calculate_orbital_wavelength_variation now in the spectrograph module of this subpackage.

With #35 we have added a new function called "imagecube2map"

There have been improvements suggested:

1. A keyword option to return Map instead of MapSequence.
2. Rename to something "nicer" e.g., simplify_to_3d or something else.

In IRISpy, the uncertainty of the spectrograph data was calculated in the reader function. The relevant parts of the code from that module were:

import astropy.units as u

# Define detector info.
DETECTOR_GAIN = {"NUV": 18., "FUV": 6.}
DETECTOR_YIELD = {"NUV": 1., "FUV": 1.5}
DN_UNIT = {"NUV": u.def_unit("DN_IRIS_NUV", 
                             DETECTOR_GAIN["NUV"] / DETECTOR_YIELD["NUV"] * u.photon),
           "FUV": u.def_unit("DN_IRIS_FUV",
                             DETECTOR_GAIN["FUV"] / DETECTOR_YIELD["FUV"] * u.photon)}
READOUT_NOISE = {"NUV": 1.2*DN_UNIT["NUV"], "FUV": 3.1*DN_UNIT["FUV"]}

# Calculate uncertainty
out_uncertainty = u.Quantity(
    np.sqrt((hdulist[window_fits_indices[i]].data * DN_unit).to(u.photon).value +
             readout_noise.to(u.photon).value**2),
    unit=u.photon).to(DN_unit).value

This function should exist in sunkit-instruments.

• The user facing function should accept a sunraster or IRISMap object, cube- or sequence-like.
• Attach the derived uncertainty to the input object in the .uncertainty attribute. In the case of a sequence-like object being input, the calculation should be performed and attached to each cube within the sequence in turn.
• The actual uncertainty formula should like in its own function.
• private versions of the function for cube and sequence objects should be written and called by the user facing one based on the input type. The cube function will also be called by the sequence one.

Describe the feature

The first parameter of split_series_using_lytaf says:

timearray ([numpy.ndarray](https://numpy.org/doc/stable/reference/generated/numpy.ndarray.html#numpy.ndarray)) – An array of times understood by sunpy.time.parse_time.

Since astropy Time objects can contain multiple times, I think it would be better for integration purposes to just accept an astropy time here instead of a numpy array of times (eww)

Proposed solution

No response

The code in the obsid module and its test module hasn't changed in the move from IRISpy. But some of the infrastructure that was used to perform the tests may need editing due to the move, e.g. missing imports, changed path names, etc.

A PR addressing this issue should not try to get tests for other modules working as they will change substantially and the test can be fixed at those times. However, a lot of the work done here may also mean there's less to do for other modules later in fixing their tests.

For simplicity, if this is the first issue addressed, this PR should comment out all other test modules by making those files giant docstrings. The modules can be uncommented as work on their modules is performed.

Open PR to iris branch.

Describe the feature

Currently get_goes_event_list just returns a list, but it looks like the data is structured enough to be converted to an astropy.table.Table.

Proposed solution

No response

In sunpy.instr.fermi, both get_detector_sun_angles_for_time() and get_detector_sun_angles_for_date() call sunpy.sun.sun functions to get the Sun's right ascension and declination. They use the functions for the apparent RA/dec referenced to the equinox of date, but I suspect that Fermi reports the spacecraft RA/dec (and hence the detectors' RA/dec) as astrometric RA/dec referenced to the J2000.0 epoch. (With sunpy/sunpy#3137, we can now provide precisely that RA/dec.) The coordinate frame used by Fermi needs to be confirmed, and the RA/dec calls changed if necessary.

Ping @aringlis

We need a document (possibly the README) that defines the scope of the package and what does and doesn't belong in it.

Rename iris_tools.py to utils/iris_utils.py.

• This must be changed in the various places that iris_tools is imported throughout the package.
• Utils for sji or spectrograph will likely be needed later so they can have their own utils files (e.g. sji_utils.py and sg_utils.py) in the utils directory at a later time.
• Most of the work on this issue will be trying to get the tests working. Tests of other modules should be allowed to fail while addressing this issue. Those tests can be fixed as part of other issues related to those modules.
• Increasing code coverage should not be a priority of any PR addressing this issue. Just getting the existing iris tools tests working will suffice. The iris tools functions may change later due to other addressing other issues.

Open PR to iris branch.

Description

The SunPy developer newcomer's guide says that intersphinx should work with Python, Numpy, and Astropy. I am guessing that for sunkit_instruments, it should also work with SunPy, as it is an affiliated package. However, when I write the restructured text Sphinx links in my code, only Numpy links are generated correctly, Astropy and SunPy links are not.

Expected vs Actual behavior

When using Sphinx links in Python code, e.g. `~numpy.ndarray` or `~sunpy.map.Map`, the expected behavior is that the links automatically point to the correct website in the documentation.

In sunkit_instruments, this only seems to work for Numpy (`~numpy.ndarray` for example gets compiled correctly), but not for e.g. `~sunpy.map.Map` or `astropy.io.fits.header.Header`.

Steps to Reproduce

See above.

System Details

• SunPy Version: 4.0.dev160+ga66cd925f
• Astropy Version: 5.0
• Python Version: 3.9.7
• OS information: Mac OS 10.16, Arch: 64bit, (i386)

The RHESSI mission maintains a catalog of flares which it has observed. Here is an example for one month.

http://hesperia.gsfc.nasa.gov/hessidata/dbase/hessi_flare_list_201309.txt

As far as I know this data is not available on the HEK.

Should we have different submodules for each instrument i.e. sunkit-instruments.goes etc?

Describe the bug

get_lytaf_events is another function that could probably return an astropy Table, instead of a numpy record array (ewwww).

To Reproduce

import sunpy

sunpy.map.Map(...)
etc

Screenshots

No response

System Details

No response

Installation method

No response

Describe the bug

The navbar on https://docs.sunpy.org/projects/sunkit-instruments/en/stable/ seems to be broken - the dropdown menus don't dropdown anything, and are just plain links.

To Reproduce

import sunpy

sunpy.map.Map(...)
etc

Screenshots

No response

System Details

No response

Installation method

No response

Describe the feature

Currently get_detector_sun_angles_for_date returns a dict that looks like:

{'n0': <Quantity 68.69273342 deg>, 'n1': <Quantity 53.92502079 deg>, 'n2': <Quantity 58.65814136 deg>, 'n3': <Quantity 54.1029331 deg>, 'n4': <Quantity 57.30596383 deg>, 'n5': <Quantity 7.86667182 deg>, 'n6': <Quantity 97.18921967 deg>, 'n7': <Quantity 114.91600048 deg>, 'n8': <Quantity 123.17782321 deg>, 'n9': <Quantity 114.29061848 deg>, 'n10': <Quantity 123.3723935 deg>, 'n11': <Quantity 172.06369333 deg>, 'time': <Time object: scale='utc' format='isot' value=2020-01-01T00:00:02.600>}

I think it would make sense to return a (single row) astropy table here. That way the object would have a nice str, and could be eaily concatenated with other values returned from this function.

Proposed solution

No response

I've just done a bunch of digging around and determined that when using get_goes_event_list, the first array coordinate is longitude and the second is latitude. This is the same as what's returned when using IDL solarsoft rd_gev (which you can drill down several levels to find gbo_struct [code], where a comment defines these two location values). This is the opposite of what you see in the GOES event lists on the NOAA web interfaces (e.g., here) where location is reported as latitude, longitude.
I noticed this when looking at several years of flare event data and checking the minmax on the two locations. The first location ranges from [-90, +90] and the second (in the timeframe I was looking) from [-32, +56]. I had serious doubts that the GOES event list had anything reported from the poles.

So my suggested change is to make variable names more explicit so people in the future don't make mistakes, or have uncertainty if they're new to solar physics, or have to go through the exercise I just did. In sunpy/instr/goes there is a function called get_goes_event_list. In there, it defines location just as it is in IDL solarsoft:
'goes_location': (r['event_coord1'], r['event_coord2'])

I suggest this be changed to goes_longitude and goes_latitude

I'm happy to make this change (or a similar one based on feedback) in a PR but I'm not sure what other code it may break.

Describe the performance issue

sunkit_instruments.fermi.download_weekly_pointing_file doesn't cache the files it downloads, instead downloading a fresh copy each time it's called. e.g. calling it twice with the same date downloads identical files to different folders:

(sunpy) 🐊 dstansby@Davids-MBP sunkit-instruments % python test.py
/var/folders/m4/v0b71w0s22z34l90psw9j82w0000gn/T/tmp41ub2_7t/lat_spacecraft_weekly_w604_p310_v001.fits
(sunpy) 🐊 dstansby@Davids-MBP sunkit-instruments % python test.py
/var/folders/m4/v0b71w0s22z34l90psw9j82w0000gn/T/tmpklqmgxz9/lat_spacecraft_weekly_w604_p310_v001.fits

It would be good if these files were cached to prevent extra downloads.

To Reproduce

import sunpy

sunpy.map.Map(...)
etc

Proposed fix

No response

A stacked calibrated event list (SCEL) is basically a visibility in a different format. The transformation from SCEL to visibly essentially only require a linear regression between the phase and count data which are already binned in the SCEL

Describe the feature

Currently despike_l1b_array inputs/outputs a bare array. For better integration with sunpy this should instead input/output a GenericMap

Proposed solution

No response

Describe the feature

The ccd_temperature takes a bare number, but is in units of temperature so should take an astropy Quantity instead.

Proposed solution

No response

Reimplement the IRISSpectrogramCube.convert_to() and IRISSpectrogramSequence.convert_to() methods as a single function that takes a sunraster cube or sequence and converts the data between counts/photons/radiance units.

• This is probably best implemented with two hidden functions. The first to act on cubes, the second on sequences by calling the cube conversion function for each cube in the sequence, and the user facing function can just be a wrapper that checks the type of the input and applies the relevant function.
• There should be some functions already in iris_tools/utils/iris_utils that may require only a little modification to be helpful in this.

Open PR to iris branch.

See our current coverage here:

https://codecov.io/gh/sunpy/sunpy/src/master/sunpy/instr/lyra.py

Describe the bug

Line 272 in sunkit_instruments.goes_xrs.goes_xrs.py reads
satellite=goests.meta.metas[0]["TELESCOP"].split()[1],
However, the key "TELESCOP" is not always present in GOES data which can lead to a KeyError.
I changed the line as follows:
satellite=goests.observatory.split('-')[1],
and the error disappeared.

To Reproduce

import sunpy.timeseries as ts
from sunkit_instruments.goes_xrs import calculate_temperature_em

goests = ts.TimeSeries('sci_gxrs-l2-irrad_g15_d20130327_v0-0-0.nc')
goests_new = calculate_temperature_em(goests)

What happened?

When I call calculate_temperature_em() on an XRSTimeSeries object, I get an error

Traceback (most recent call last):
File "", line 1, in
File "", line 5, in
File "/home/bulat/anaconda3/envs/sunpy4/lib/python3.10/site-packages/sunkit_instruments/goes_xrs/goes_xrs.py", line 272, in calculate_temperature_em
satellite=goests.meta.metas[0]["TELESCOP"].split()[1],
File "/home/bulat/anaconda3/envs/sunpy4/lib/python3.10/site-packages/sunpy/util/metadata.py", line 164, in getitem
return OrderedDict.getitem(self, key.lower())
KeyError: 'telescop'

Expected behavior

A new XRSTimeSeries object should be created with temperature and emission measure calculated.

Screenshots

No response

System Details

sunpy 4.0.0
astropy 5.1
python 3.10.4
system Ubuntu (18.04, Linux 4.15.0-180-generic)

Installation method

conda

Write equivalencies using astropy units from counts (u.ct) to photons for iris_sji_dn_equivalency, iris_fuv_dn_equivalency and iris_nuv_dn_equivalency.

• Currently there is no DN unit (data number) in astropy so make the equivalency between counts (u.ct) and photons.
• The SJI equivalency should be in the sji_utils.py file while the FUV and NUV equivalencies should be in the sg_utils.py file.
• Look up astropy equivalencies to see how to write new equivalencies.
• The conversions between DN and photons via the detector gains can be found in the definition of the DN units in iris_tools.
• These equivalencies should replace the IRIS units.

Open PR to IRIS branch.

dependencies on zeep and drms packages are missing

To Reproduce

from sunpy.net import Fido

System Details

Google Colab (Ubuntu 20.04.6 LTS)

Installation method

!pip install sunpy

Describe the bug

from sunkit_instruments.fermi import download_weekly_pointing_file, get_detector_sun_angles_for_time, plot_detector_sun_angles
from astropy.visualization import time_support, quantity_support

time_support()
quantity_support()

f = download_weekly_pointing_file('2020-01-01')
angles = get_detector_sun_angles_for_time('2020-01-01', f)
plot_detector_sun_angles(angles)

fails with

% python test.py
Traceback (most recent call last):
  File "/Users/dstansby/mambaforge/envs/sunpy/lib/python3.10/site-packages/astropy/utils/shapes.py", line 219, in __getitem__
    return self._apply('__getitem__', item)
  File "/Users/dstansby/mambaforge/envs/sunpy/lib/python3.10/site-packages/astropy/time/core.py", line 1119, in _apply
    jd1 = apply_method(jd1)
IndexError: too many indices for array: array is 0-dimensional, but 1 were indexed

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/dstansby/github/sunkit-instruments/test.py", line 9, in <module>
    plot_detector_sun_angles(angles)
  File "/Users/dstansby/github/sunkit-instruments/sunkit_instruments/fermi/fermi.py", line 242, in plot_detector_sun_angles
    plt.xlabel("Start time: " + angles["time"][0].isoformat())
  File "/Users/dstansby/mambaforge/envs/sunpy/lib/python3.10/site-packages/astropy/utils/shapes.py", line 222, in __getitem__
    raise TypeError('scalar {!r} object is not subscriptable.'
TypeError: scalar 'Time' object is not subscriptable.

Perhaps I'm doing something wrong, but if so there needs to be better documentation on what the valid input types are for plot_detector_sun_angles.

To Reproduce

import sunpy

sunpy.map.Map(...)
etc

Screenshots

No response

System Details

No response

Installation method

No response

A standard data product from rhessi are "corrected" count rates. These rates remove a number of instrumental affects from the summary count rate product. The implementation exists in IDL and can be found here http://hesperia.gsfc.nasa.gov/ssw/hessi/idl/qlook_archive/hsi_obs_summ_rate__define.pro. All of the information necessary to perform this exists in the observing summary files which can be downloaded using sunpy.instr.rhessi.get_obssumm_file(). Corrected count rates should be added to the output of sunpy.instr.rhessi.parse_obssumm_file() function.

Reimplement apply_dust_mask method on IRIS Map classes as functions that take IRIS Map objects. This is probably best implemented as with two hidden functions. The first to act on cubes, the second on sequences by calling the cube dust mask function for each cube in the sequence, and the user facing function can just be a wrapper that checks the type of the input and applies the relevant function.

Open PR to IRIS branch

Describe the feature

Following discussion with @alasdairwilson it would be nice to have the goes_xrs submodule be able to download the GEOS NRT json file and return a timeseries

something along the lines of

from sunkit_instruments import goes_xrs
goes_nrt = goes_xrs.get_nrt()

As this package is new, some instrument subpackages may need rapid development and slightly looser standards on individual PRs. Therefore, in order that multiple haphazard instrument-specific development efforts do not turn the package into an unmanageable mess, development should be modularized between instruments.

Proposed Procedure: there should be instrument branches on which people can collaborate and contribute code for a specific instrument, e.g. iris. The contribution procedure would then be the following for different scenarios:

Implementation of a new feature or roadmap for a certain instrument

• New instrument-specific feature developed.
• Developer opens PR to instrument branch.
• PR is iterated on and merged.
• Subsequent PRs opened and merged if feature or roadmap requires several.
• Once feature(s) implemented, passing tests, and ready for release, open PR to master branch. This reduces the amount of buggy or incomplete code making into master. This minimizes the chance of the release of the whole package being held up by one instrument.

Feature required for whole package, e.g. retemplating

• Open PR directly to master

Bugfix on instrument branch not yet in master

• Open PR to instrument branch

Bugfix on master for specific instrument

• Open PR directly to master.
• Update instrument branch by pulling master and resolving any conflicts.
  If bug-fix, PR can be cherry-picked/"backported" to master.

Need for change outside of instrument sub-packages discovered as part of instrument specific development.

• Separate package and instrument-specific features in different PRs where the package PR is to master and the instrument PR is to the instrument branch.
• If necessary, merge developer master branch into developer instrument branch to continue work.
• Merge package PR first, and update instrument branch before merging new feature to instrument branch.

Description

As discussed in GOES-R XRS user's guide, the operational data prior to GOES-16 had scaling factors applied by SWPC to adjust the GOES 8-15 irradiances to match fluxes from GOES-7. To get true irradiances from the operational data, the scaling factors of 0.85 (for the XRS-A channel) and 0.7 (for the XRS-B channel) applied to GOES 8-15 had to be removed. There are no such scaling factors in the GOES-R XRS data; the provided irradiances are in true physical units.

This is creating problem in calculating Temperature and Emission Measure from the routine "_goes_chianti_tem".
SSW's "goes_chianti_tem.pro" has been modified to handle this inconsistency by introducing "remove_scaling" flag. Following is the modification comment by Kim Tolbert on 2020-Jun-01.

;added remove_scaling keyword. G8-15 old data (from SDAC or Yohkoh)
; archives was scaled to match G1-7 data. Here we are unscaling it to get back to physical
; units before we use transfer functions. However new NOAA data (G16,17, and reprocessed G13-15)
; has not had that scaling applied so we do not need to modify it here. So remove_scaling
; should be:
; 0 for G16,17, and reprocessed G13-15 from NOAA
; 1 for G8-12 and earlier versions of G13-15
; doesn't matter for G1-7

Something similar needs to be added to handle GOES-R series data.

Description

I know we are planning to move the instr out of core, but I'm playing around with sunpy.instr. calculate_temperature_em(), and there seems to be a bug with the data manager to get the required chianti files.

Expected behavior

>>> goes_ts = TimeSeries('go1520120514.fits').truncate('2012-05-14 13:32', '2012-05-14 13:45')
>>> ts_new = goes.calculate_temperature_em(goes_ts)   

and this should return a lovely timeseries with two extra columns for temperature and emission measure

Actual behavior

/Users/laurahayes/anaconda3/lib/python3.7/functools.py:824: UserWarning: Discarding nonzero nanoseconds in conversion
  return dispatch(args[0].__class__)(*args, **kw)
Files Downloaded: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  8.08file/s]
WARNING: SunpyUserWarning: [Errno 2] No such file or directory: '/Users/laurahayes/sunpy/data_manager/goes_chianti_temp_cor.csv' [sunpy.data.data_manager.cache]
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-9-e8e77b0651c9> in <module>
----> 1 ts_new = goes.calculate_temperature_em(goes_ts)

~/sunpy_dev/sunpy-git/sunpy/instr/goes.py in calculate_temperature_em(goests, abundances, download, download_dir)
    268         satellite=goests.meta.metas[0]["TELESCOP"].split()[1],
    269         date=goests.to_dataframe().index[0],
--> 270         abundances=abundances, download=download, download_dir=download_dir)
    271 
    272     ts_new = timeseries.XRSTimeSeries(meta=copy.deepcopy(goests.meta),

~/anaconda3/lib/python3.7/site-packages/astropy/units/decorators.py in wrapper(*func_args, **func_kwargs)
    232             # Call the original function with any equivalencies in force.
    233             with add_enabled_equivalencies(self.equivalencies):
--> 234                 return_ = wrapped_function(*func_args, **func_kwargs)
    235             if wrapped_signature.return_annotation not in (inspect.Signature.empty, None):
    236                 return return_.to(wrapped_signature.return_annotation)

~/sunpy_dev/sunpy-git/sunpy/instr/goes.py in _goes_chianti_tem(longflux, shortflux, satellite, date, abundances, download, download_dir)
    418     temp = _goes_get_chianti_temp(fluxratio, satellite=satellite,
    419                                   abundances=abundances, download=download,
--> 420                                   download_dir=download_dir)
    421     em = _goes_get_chianti_em(longflux_corrected, temp, satellite=satellite,
    422                               abundances=abundances, download=download,

~/sunpy_dev/sunpy-git/sunpy/data/data_manager/manager.py in wrapper(*args, **kwargs)
     78                         if self._cache_has_file(urls):
     79                             raise ValueError(" Hash provided does not match the hash in database.")
---> 80                         file_path = self._cache.download(urls)
     81                         if hash_file(file_path) != sha_hash:
     82                             # the hash of the file downloaded does not match provided hash

~/sunpy_dev/sunpy-git/sunpy/data/data_manager/cache.py in download(self, urls, redownload)
     84                 return Path(details['file_path'])
     85 
---> 86         file_path, file_hash, url = self._download_and_hash(urls)
     87 
     88         self._storage.store({

~/sunpy_dev/sunpy-git/sunpy/data/data_manager/cache.py in _download_and_hash(self, urls)
    169                 warn(e, SunpyUserWarning)
    170         else:
--> 171             raise RuntimeError("Download failed")

RuntimeError: Download failed

Where is the bug?

I'm pretty confused about where the bug is lying - where is the hash key generated here?

@manager.require('file_temp_cor',
                 [urljoin(GOES_REMOTE_PATH, FILE_TEMP_COR)],
                 '3d8ddaaabf0faf75ba8d15e0c468896ce3d7622cc23076bf91437951e0ab3ad4')

The url exists and can be downloaded, the data manager is hurting my head 🤯

This issue proposes what the IRIS submodule of this package should look like.

• This subpackage should replace the IRIS-specific functionality in sunraster with the exception of the spectrograph reader functions.
• For raster data, it should NOT provide raster classes, but instead provide functions that act on sunraster objects.
• The iris submodule should provide the following classes:
  • IRISMapCube
  • IRISMapCubeSequence (Consider renaming to IRISMapSequence?)
  • ObsId
• iris submodule should NOT provide the following classes and functions:
  • IRISSpectrograph
  • IRISSpectrogramCubeSequence
  • IRISSpectrogramCube
  • calculate_exposure_time_correction and uncalculate_exposure_time_correction. (These are now handled by sunraster.)
  • _convert_iris_sequence (Not used anywhere. Double check this.)
• The following methods from IRISMapCube/Sequence and IRISSpectrogramCube/Sequence classes should be provided as functions that take Raster and RasterSequence objects from sunraster:
  • IRISSpectrogramCube.convert_to() and IRISSpectrogramSequence.convert_to() -> convert_iris_spectrogram_to() (or better named function).
    • This function should accept an IRISSpectrogramCube or IRISSpectrogramSequence, convert the data, and return a new instance of the input class.
    • This will require private versions of the function of the Cube and Sequence cases.
  • IRISMapCube.convert_to() and IRISMapSequence.convert_to() -> convert_iris_map_to() (or better named function).
    • This function should accept an IRISMapCube or IRISMapSequence, convert the data, and return a new instance of the input class.
    • This will require private versions of the function of the Cube and Sequence cases.
• iris subpackage should contain the following modules:
  • obsid.py
    • ObsId
  • spectrograph.py
    • convert_iris_spectrogram
    • calculate_orbital_wavelength_variation
    • FUV and NUV DN unit equivalencies
  • sji.py
    • convert_iris_map
    • apply_dust_mask
    • calculate_dust_mask
    • read_iris_sji_level2_fits
    • SJI DN unit equivalency
  • utils.py (for utils that are independent of whether data is from SJI or SG.)
    • get_iris_response
    • fit_iris_xput
    • get_detector_type
    • convert_between_DN_and_photons
    • convert_or_undo_photons_per_sec_to_radiance
    • calculate_uncertainty (Take a sunraster or IRISMap object and calculate the uncertainty based on the data and some detector properties. Attach to the input object's uncertainty attribute.

Some tools have not been mentioned here but should also be included in the most logical place if above functions depend on them. Any functions not used anywhere should be considered for deletion.

When this submodule or features within it can be deprecated once the instrument-team-backed IRISpy package is released.

Sometimes a selection of the goes tests fail:

============================================================================= short test summary info ==============================================================================
FAILED ../../.tox/py38-online/lib/python3.8/site-packages/sunkit_instruments/goes_xrs/tests/test_goes_xrs.py::test_calc_rad_loss_nokwags - ValueError: ['http://hesperia.gsfc.nas...
FAILED ../../.tox/py38-online/lib/python3.8/site-packages/sunkit_instruments/goes_xrs/tests/test_goes_xrs.py::test_calc_rad_loss_obstime - ValueError: ['http://hesperia.gsfc.nas...
========================================================= 2 failed, 18 passed, 205 warnings, 5 rerun in 397.99s (0:06:37) ==========================================================

What ones or when they fail seems to be random(?!) but they all use the data manager in some form.

Describe the bug

I tried to calculate the temperature and corresponding volume emission measure using GOES-17 satellite according
for event 15 Jan 2022 (06:50 - 07:20) using the GOES-17 data.

But it seems to me that for this instrument the programme "sunkit_instruments.goes_xrs.calculate_temperature_em" needs some additional key, which are not given in that Guide. You can calculate temperature and emission measure only for GOES-15 according User Guide.

See the program and errors below:

import matplotlib.pyplot as plt

import numpy as np

import datetime

from sunpy import timeseries as ts

from sunpy.net import Fido

from sunpy.net import attrs as a

from sunkit_instruments.goes_xrs import calculate_temperature_em

results_17 = Fido.search(a.Time("2022-01-15 06:50", "2022-01-15 07:20"), a.Instrument("XRS"),

a.goes.SatelliteNumber(17))

files = Fido.fetch(results_17, concatenate=True)

goests = ts.TimeSeries(files)

goests_new = calculate_temperature_em(goests)

I see error:
File M:\anaconda3\lib\site-packages\sunkit_instruments\goes_xrs\goes_xrs.py:272 in calculate_temperature_em
satellite=goests.meta.metas[0]["TELESCOP"].split()[1],

File M:\anaconda3\lib\site-packages\sunpy\util\metadata.py:164 in getitem
return OrderedDict.getitem(self, key.lower())

KeyError: 'telescop'

To Reproduce

import matplotlib.pyplot as plt

import numpy as np

import datetime

from sunpy import timeseries as ts

from sunpy.net import Fido

from sunpy.net import attrs as a

from sunkit_instruments.goes_xrs import calculate_temperature_em

results_17 = Fido.search(a.Time("2022-01-15 06:50", "2022-01-15 07:20"), a.Instrument("XRS"),

a.goes.SatelliteNumber(17))

files = Fido.fetch(results_17, concatenate=True)

goests = ts.TimeSeries(files)

goests_new = calculate_temperature_em(goests)

Screenshots

No response

System Details

Sunpy 4.0.5

Installation method

conda

Describe the feature

Hi everyone!

As discussed in the Element Sunpy group on 17-Oct-2023, I was wondering if there is a way to convolve Sunpy maps with externally provided functions. Specifically, I have an HMI map and for an optical design I'm working on, I need to convolve it with an artificial PSF of 10 arcsec FWHM. It seems that we cannot do this directly in Sunpy, but we can follow these steps: extract the array from the Sunpy map, apply scipy.ndimage.gaussian_filter, and then reconstruct the map.

This seems to be a common task, so it may be useful to have a direct method for doing it in Sunpy.

Proposed solution

A possible solution would be to create a method including the following steps: extract the array from the Sunpy map, apply scipy.ndimage.gaussian_filter, and then reconstruct the map.