The example notebooks need some additional textual explanation to be understandable and useful to a new user. Please also check for spelling mistakes throughout (e.g. chuck->chunk, obserbed->observed). More importantly,

guidestar_targets.ipynb:
This notebook in particular needs a fair bit of explanation to be useful. There is no description of what the observations were doing, why the guide star changed so many times, and what utility there is in the demonstrated figures (what do they show, how could they be used). You've also written that you're going to use PID 2079 but actually use PID 1534. You also included an actual mast_api_token in Cell [6] which is a really bad idea (thankfully the API token is 5 months old now and will have already expired). You should instead be loading this from an external file/variable using something like spk.mast_api_token = os.environ.get('MAST_API_TOKEN') to ensure this never happens.

pixel_centroid_mnemonics.ipynb:
Some context at the top of the notebook describing what the notebook is going to teach folks would be very helpful. Also briefly explaining the observations of PID 2079 would help set the context some. I'm not really sure what is meant by "We can see that there are chucks of data points that largely changes in flux for each chunk." The outputs of In [24] and In [15] seem to show that the time-axis may not be sorted, resulting in some instances where there are two lines for a given x value (because pyplot by default is just connecting the dots in the order they were provided); this may hint at a more concerning underlying issue with the ordering of the values gathered by spelunker. Either way, its probably best to use points instead of connected lines to more clearly demonstrate the breaks between observations.

pixel_centroid_mnemonics.rst:
There seems no need to me to have an rst file in addition to the notebook, and all of the figures that went along with the notebook are broken in this rst version.

Also, note that some of the outputs (in particular the tables) and even some of the inputs extend too far on your readthedocs page, making them unreadable at times.

This issue was made as part of the review for openjournals/joss-reviews#6202

The "Python Module Index" found at https://jwst-fgs-spelunker.readthedocs.io/en/latest/py-modindex.html which is accessed through clicking on the MODULES link at the top of the readthedocs page only shows spelunker, and when you click on that it just takes you to https://jwst-fgs-spelunker.readthedocs.io/en/latest/optimizing_photometry.html#module-spelunker.

This issue was made as part of the review for openjournals/joss-reviews#6202

There is something strange going on with the formatting of your API documentation on readthedocs for the load, download, and readfile functions. For only these three functions, the documentation for the function arguments overflows the width of the expected text area. Not sure why this is happening, but my guess is that it is because you used a different docstring formatting for these three functions compared to all your other functions.

You do not need to resolve this issue before the acceptance of your JOSS paper - just wanted to point this issue out to you since resolving this will make your API look cleaner

Ran into this problem trying the latest version:

Traceback (most recent call last):
  File "<ipython-input-1-460cda62bfbc>", line 3, in <module>
    spk = spelunker.load(pid=2221)
  File "/home/adam/anaconda3/envs/python3.9/lib/python3.9/site-packages/spelunker/spelunker.py", line 86, in __init__
    self.download(pid, obs_num=obs_num, visit=visit, visit_group=visit_group, parallel_sequnence_id=parallel_sequnence_id,
  File "/home/adam/anaconda3/envs/python3.9/lib/python3.9/site-packages/spelunker/spelunker.py", line 384, in download
    reformed_directory = np.char.add(np.char.add(fg_table['sliced_directory'], f_slash), fg_table['filenames'])
  File "<__array_function__ internals>", line 180, in add
  File "/home/adam/anaconda3/envs/python3.9/lib/python3.9/site-packages/numpy/core/defchararray.py", line 310, in add
    return _vec_string(arr1, (dtype, out_size), '__add__', (arr2,))
TypeError: string operation on non-string array

you should probably use os.path.join rather than np.char.add to join path components

Below is some feedback on your code itself:

The following snippet could be greatly simplified in the following way:

bool_mask = fg_flux >= 0

The astropy.table.Table.sort function operates on the existing table and does not return a new table, so there's no need to capture the returned None value (and same goes for every other time you call sort)

The code in the following lines is used in all the if/elif/else clauses, so it should be moved before the if statement to avoid repeated code and to make maintaining the code easier:

Rather than using the code in this snippet, you could much more quickly and easily determine the brightest few pixels by doing

mask = np.zeros_like(frame.flatten())
good_inds = np.argsort(frame.flatten(), axis=None)[-npixels:]
mask[good_inds] = 1
return mask.reshape(frame.shape)

In the following snippets, the and operator is not working the way that you intend it to. For example, in the first snippet instead of determining whether both fg_flux and fg_time are of type str, the code is actually doing if (type(fg_flux)) and (type(fg_time)==str) which is definitely not the intended meaning of the code since this will return True as long as type(fg_flux) is not "falsy" which I don't think it will ever be. These are just the few cases where I noticed you made this mistake, you should look through the code to see if there are any other places where you've done this.

Why is the MAST API token sometimes passed in as a function parameter but assumed to be an attribute for the following snippet? I recommend sticking with one consistent interface, either only offering it as a parameter for all functions, only offering it as an attribute for all functions, or offering both options for all functions. Additionally, I believe that doing val == None is risky and discouraged; I believe you should instead be doing val is None throughout your code.

For the following two functions, it would make much more sense to me to allow plotting if an input parameter was set to True but not force people to plot just to access the parameter. Additionally, I would think the retrieved values should be returned from the function; that way, someone doesn't pass in "SA_ZHGAUPST" and then not know how to access the values without reading the docs and seeing that they have to then access spk.mnemonics_event_hga

Why not save the estimated parameters (like centroids and PSF widths) if self.quickfit_results is not None?

This issue was made as part of the review for openjournals/joss-reviews#6202

You should add a sentence or two describing what the Guidestar positions figure shows, as it is not immediately clear to me let alone someone new to JWST data. Describe what the symbols mean and tersely describe why JWST would be changing between different guide stars. Any guide stars are also presumably covered by your "x" symbols - perhaps an additional figure (or more than one) or subfigure showing cutouts around the guide stars would be helpful for some use cases?

This terse description should be added to the README.md file as well as the docs/simple_quickstart.rst file

This issue was made as part of the review for openjournals/joss-reviews#6202

Here's the error I'm getting when attempting to perform a Gaussian fit for a specific FGS dataset (it seems to work for other datasets). I'm using macOS 13.6.6, python 3.10.14 and spelunker 1.1.17

32%|███▏ | 97909/308130 00:57<02:50, 1232.69it/s [2024-05-24 16:10:29,760 E 75152 10353252] (raylet) file_system_monitor.cc:111: /tmp/ray/session_2024-05-24_16-08-56_941025_75129 is over 95% full, available space: 13070712832; capacity: 994662584320. Object creation will fail if spilling is required.
35%|███▍ | 107579/308130 [01:05<02:02, 1631.54it/s]
Traceback (most recent call last):
File "/Users/matthewhooton/PycharmProjects/spelunker/run.py", line 68, in
_ = spk.gauss2d_fit(ncpus=4)
File "/Users/matthewhooton/anaconda3/envs/spelunker/lib/python3.10/site-packages/spelunker/spelunker.py", line 1250, in gauss2d_fit
popt3 = ray.get(i)
File "/Users/matthewhooton/anaconda3/envs/spelunker/lib/python3.10/site-packages/ray/_private/auto_init_hook.py", line 21, in auto_init_wrapper
return fn(*args, **kwargs)
File "/Users/matthewhooton/anaconda3/envs/spelunker/lib/python3.10/site-packages/ray/_private/client_mode_hook.py", line 103, in wrapper
return func(*args, **kwargs)
File "/Users/matthewhooton/anaconda3/envs/spelunker/lib/python3.10/site-packages/ray/_private/worker.py", line 2623, in get
values, debugger_breakpoint = worker.get_objects(object_refs, timeout=timeout)
File "/Users/matthewhooton/anaconda3/envs/spelunker/lib/python3.10/site-packages/ray/_private/worker.py", line 861, in get_objects
raise value.as_instanceof_cause()
ray.exceptions.RayTaskError(TypeError): ray::fit_gaussian() (pid=75159, ip=127.0.0.1)
File "/Users/matthewhooton/anaconda3/envs/spelunker/lib/python3.10/site-packages/spelunker/spelunker.py", line 1211, in fit_gaussian
initial_guess[1], initial_guess[2] = int(coords[1]), int(coords[0])
TypeError: only length-1 arrays can be converted to Python scalars

Because the gauss2d_fit function is so slow, I can't tell if the function call has crashed or it is just taking a long time since there is no progress bar. I can submit a pull request to add a progress bar using tqdm which I've already got setup on my local branch which partially resolves this issue.

However, in investigating this issue further I noticed that you're currently using ray to parallelize each individual fit rather than running multiple fits in parallel. Each individual fit takes far less than a second, so this method of parallelization offers little-to-no gains (in fact, using a tool like htop I can see that regardless of what I set ncpus to, the function only really uses ~1 thread since it finishes too fast to use any more threads). For reference when running the fgs-spelunker-and-tsos.ipynb demo notebook, the loop at line spelunker.py#L1260 took me 8m4s with the full 28 threads available on my CPU (a desktop Intel i7-14700K) and 7m51s with only 1 thread, while the loop at spelunker.py#L1272 took 18m46s with 28 threads and 19m04s with only 1 thread. What you need to do is setup an asynchronous collection of threads that can fit individual frames in parallel which should offer roughly linear scaling in speed-up with the number of threads used (that is, 2 threads should give ~2x speed-up, and 10 threads should give ~10x speed-up), while my testing above showed basically no change in speed with 28 threads vs 1 thread. I'm not immediately sure how to do this with ray but have done this with Python's built-in package multiprocessing over in Eureka (see for example, eureka.S3_data_reduction.background.BGsubtraction()).

As a reminder, this is not really related to the review of your JOSS paper and doesn't need to be resolved before that paper is accepted. I'm just now an invested future user wanting to help improve a wonderfully useful package!

The jwstuser installation instructions are missing from the installation.rst docs file. This is described in your README.md file, but should also be described on your installation instructions page

This issue was made as part of the review for openjournals/joss-reviews#6202

I get this warning every time I load spelunker:
Warning: jwstuser is not installed. mnemonics() will not work.

While jwstuser is not part of spelunker I wasn't able to find any information in the documentation and online. I assume this will be a very common warning message so I recommend adding some more details on that or link to the relevant pages.

This issue was made as part of the review for openjournals/joss-reviews/issues/6202.

Below is some feedback on your API documentation page. I've made this a check list so that it's easy for me to cross things off after you've made some updates.

• Please ensure that all functions have all of their inputs and (if relevant) outputs documented with a description of the variables and their data types. You've done this for many but not all functions. To save you some time, in some places you can just copy-paste the documentation for parameters that are already documented in the load function.
• There is a missing space between the parameter names and parameter types in the documentation for basically every function except for load
• In gauss2d_fit, you should explain briefly what all of the different parameters mean (either in text or by linking to some astropy page or whatever), since a newcomer might be confused by parameters like theta and offset.
• Check for typos throughout (e.g. "speficied"->"specified")
• It seems unusual to me to force the plotting of the mnemonics in the mnemonics and mnemonics_local functions. Having a plot input boolean parameter would seem more appropriate. Additionally, is the mnemonics_event attribute set by the mnemonics function some kind of a dictionary that will store multiple mnemonics or is it something that'll be overwritten every time the mnemonics function is called with a new parameter?
• There is a confusing mixture of object-oriented programming and functional programming. For example, some functions like negative_flux_preprocessing require that data arrays be passed in, while functions like normalization_flux_preprocessing seem not to accept inputs and instead work on the internally stored arrays, while yet other functions like gauss2d_fit are a hybrid and will either operate on the internally stored arrays or on an array provided as an input. There should be more consistency in what you allow for each function to help the users gain some intuition for how to use your code.
• It is likely also best that the function periodogram not force plotting but offer it as a (default on) option.
• How is the binning inside timeseries_binned_plot controlled? Why not make parameters like bin size available to the user?
• The documented parameters for timeseries_list_plot are not in the same order as they are input into the function.

This issue was made as part of the review for openjournals/joss-reviews#6202

I found what I think is a small bug in spk.guidestar_plot() in the event that only 1 guide star position is returned for an observation. You can test this on my data (publicly available: pid=3730, obs_num='13', visit='1').

I was able to fix this by adding some code just after after line

I'm sure there are other ways to fix this too.

Thank you for developing such a cool code for looking at FGS data!

I tried loading spelunker with some cycle 2 JWST proposal IDs (that have definitely been completed) like this:
spk = spelunker.load(pid=4098, obs_num='13', visit='1', save=True)

but it gives an error message:

WARNING: NoResultsWarning: Query returned no results. [astroquery.mast.discovery_portal]
2024-01-30 13:39:17,619 - stpipe - WARNING - NoResultsWarning: Query returned no results.

Was there a recent update that affected the newer JWST observations?

This issue was made as part of the review for openjournals/joss-reviews/issues/6202

At present, your documentation at https://jwst-fgs-spelunker.readthedocs.io/ is poorly optimized for smaller screens (e.g. laptops and especially phones), requiring lots of horizontal scrolling. This must just be caused by the particular template that you have chosen since most other readthedocs pages that I've seen adapt very well to differently sized screens (e.g. https://eurekadocs.readthedocs.io/)

This issue was made as part of the review for openjournals/joss-reviews#6202

Hi @GalagaBits,

Here's an issue I think needs to be resolved. When spelunker does it magic, it seems it stays forever stuck on the spelunker_outputs folder, and never goes back to the working directory on which analyses were started. This is a bit problematic for usages in notebooks, because the folder changes dynamically as you use spelunker.

An easy fix I belive is to either:

1. Don't change directories, simply output products into that directory without jumping to it.
2. Come back to the original directory every time we jump into the spelunker_outputs folder.

Hope this makes sense. Thanks!

N.

Below is some feedback on your (very helpful!) fgs-spelunker-and-tsos.rst documentation page. I've made this a check list so that it's easy for me to cross things off after you've made some updates.

• The program version on your readthedocs page is currently set to Program version: ver. 0.5.3 which doesn't exist on GitHub and is greatly out-of-date with the pip installed version 1.1.0. The page should either be remade with the newer code version or the code version should be updated on the page.
• targetted -> targeted
• accreeting -> accreting
• HAT-P-14~b -> HAT-P-14 b
• You should briefly describe where the 'hp14_lightcurve.dat' and 'w39_lightcurve.dat' files comes from (citing the source paper and software as appropriate) and explain what package(s) the reader could use to make such a file themselves.
• Adding plt.show() at the end of blocks of plotting code will avoid the printing of values like (0.0, 6.09976451843977) to the console
• The transit appears to start closer to 2.75 hours after the start, not 2.5
• It would be useful to briefly describe where someone might find the pid, obs_num, and visit values for an observation they want to analyze. Newcomers may not immediately be familiar with these values.
• The raw photometry from the guidestar varies by a lot more than our target star. I assume much of this variation is just photon noise given the very short exposure time - if this is the case, it would be useful to briefly describe this to remind the user that the cadence varies between the science instrument and the FGS and to explain why temporal binning would be helpful.
• Rather than making a custom bin_fgs_to_science function from scratch, wouldn't it be better to make use of a spelunker function to get users familiar with more of the utilities you have built?
• The shown outputs from running spk.gauss2d_fit(ncpus=4) include a large number of error messages. Ideally these should be resolved if possible, they captured and hidden if unimportant, or at a minimum they should be explained in the tutorial so that folks know that these kinds of errors are not concerning. Additionally, there should likely also be either a mute or verbose option added so that the printed table does not have to be output to the console.
• ..."appears to be a small tilt event." You should briefly explain what a tilt event is and add a link to a relevant JDox page
• "Best of all, we can see the event at any resolution we want thanks to it." Probably best to remind the reader again here that the FGS data have a finer time sampling which is what allows this higher temporal resolution (also likely worth stating temporal resolution to differentiate from spectral resolution)
• Rather than making a new bin_data function, you should use your built-in spelunker.bin_data function
• "Interestingly, in this case, the Y-standard deviation samples the event even better." This is not obvious from the plots. You might try inverting the secondary y-axis for sigma_y and/or plotting both sigma_x and sigma_y after some kind of normalization
• Finally, it would be worth adding a concluding sentence or few that would explain where someone might go from here. Something about how one could use these FGS data as covariates when fitting their lightcurves to remove noise would be useful (e.g. linearly decorrelate against FGS y-pixel position).

This issue was made as part of the review for openjournals/joss-reviews#6202

The tests are currently failing for v1.1.0 with Python 3.11.0 (see traceback below).

> python general_test.py
Warning: jwstuser is not installed. mnemonics() will not work.

 >> Testing spelunker version  1.1.0
Current working directory for spelunker: /home/taylor/Refereeing/Spelunker/JWST-FGS-Spelunker/testing/spelunker_outputs

Connecting with astroquery...
Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:JWST/product/jw01541001001_gs-fg_2022159061621_cal.fits to /home/taylor/Refereeing/Spelunker/JWST-FGS-Spelunker/testing/spelunker_outputs/mastDownload/JWST/jw01541001001_02101_00004-seg001_nis/jw01541001001_gs-fg_2022159061621_cal.fits ...
|==================================================================================| 2.5M/2.5M (100.00%)         0s
Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:JWST/product/jw01541001001_gs-fg_2022159062424_cal.fits to /home/taylor/Refereeing/Spelunker/JWST-FGS-Spelunker/testing/spelunker_outputs/mastDownload/JWST/jw01541001001_02101_00004-seg001_nis/jw01541001001_gs-fg_2022159062424_cal.fits ...
|==================================================================================| 7.5M/7.5M (100.00%)         1s
Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:JWST/product/jw01541001001_gs-fg_2022159132423-seg001_cal.fits to /home/taylor/Refereeing/Spelunker/JWST-FGS-Spelunker/testing/spelunker_outputs/mastDownload/JWST/jw01541001001_02101_00004-seg001_nis/jw01541001001_gs-fg_2022159132423-seg001_cal.fits ...
|==================================================================================|  51M/ 51M (100.00%)         5s
Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:JWST/product/jw01541001001_gs-fg_2022159132423-seg002_cal.fits to /home/taylor/Refereeing/Spelunker/JWST-FGS-Spelunker/testing/spelunker_outputs/mastDownload/JWST/jw01541001001_02101_00004-seg001_nis/jw01541001001_gs-fg_2022159132423-seg002_cal.fits ...
|==================================================================================|  51M/ 51M (100.00%)         6s
Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:JWST/product/jw01541001001_gs-fg_2022159132423-seg003_cal.fits to /home/taylor/Refereeing/Spelunker/JWST-FGS-Spelunker/testing/spelunker_outputs/mastDownload/JWST/jw01541001001_02101_00004-seg001_nis/jw01541001001_gs-fg_2022159132423-seg003_cal.fits ...
|==================================================================================|  51M/ 51M (100.00%)         5s
Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:JWST/product/jw01541001001_gs-fg_2022159132423-seg004_cal.fits to /home/taylor/Refereeing/Spelunker/JWST-FGS-Spelunker/testing/spelunker_outputs/mastDownload/JWST/jw01541001001_02101_00004-seg001_nis/jw01541001001_gs-fg_2022159132423-seg004_cal.fits ...
|==================================================================================|  51M/ 51M (100.00%)         5s
Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:JWST/product/jw01541001001_gs-fg_2022159132423-seg005_cal.fits to /home/taylor/Refereeing/Spelunker/JWST-FGS-Spelunker/testing/spelunker_outputs/mastDownload/JWST/jw01541001001_02101_00004-seg001_nis/jw01541001001_gs-fg_2022159132423-seg005_cal.fits ...
|==================================================================================|  51M/ 51M (100.00%)         5s
Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:JWST/product/jw01541001001_gs-fg_2022159132423-seg006_cal.fits to /home/taylor/Refereeing/Spelunker/JWST-FGS-Spelunker/testing/spelunker_outputs/mastDownload/JWST/jw01541001001_02101_00004-seg001_nis/jw01541001001_gs-fg_2022159132423-seg006_cal.fits ...
|==================================================================================|  51M/ 51M (100.00%)         6s
Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:JWST/product/jw01541001001_gs-fg_2022159132423-seg007_cal.fits to /home/taylor/Refereeing/Spelunker/JWST-FGS-Spelunker/testing/spelunker_outputs/mastDownload/JWST/jw01541001001_02101_00004-seg001_nis/jw01541001001_gs-fg_2022159132423-seg007_cal.fits ...
|==================================================================================|  51M/ 51M (100.00%)         5s
Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:JWST/product/jw01541001001_gs-fg_2022159132423-seg008_cal.fits to /home/taylor/Refereeing/Spelunker/JWST-FGS-Spelunker/testing/spelunker_outputs/mastDownload/JWST/jw01541001001_02101_00004-seg001_nis/jw01541001001_gs-fg_2022159132423-seg008_cal.fits ...
|==================================================================================|  35M/ 35M (100.00%)         3s
2024-01-29 14:29:28,303 INFO worker.py:1724 -- Started a local Ray instance.

 >> Results:

         - Downloading & storage of data (spk.load)........PASS
         - 2D gaussian fits (spk.gauss2d_fit)........PASS
         - Guidestar plot (spk.guidestar_plot)........FAIL

         - Final assessment: some tests failed. Here is a summary of those, along with errors raised:

                 1) Guidestar plot (spk.guidestar_plot) failed with a IndexError :

Traceback (most recent call last):
  File "/home/taylor/Refereeing/Spelunker/JWST-FGS-Spelunker/testing/general_test.py", line 72, in test_guidestarplot
    spk.guidestar_plot()
  File "/home/taylor/miniconda3/envs/spelunker/lib/python3.11/site-packages/spelunker/spelunker.py", line 1483, in guidestar_plot
    ax, hdu = plot_finder_image(target, survey='DSS', fov_radius=fov_radius,)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/taylor/miniconda3/envs/spelunker/lib/python3.11/site-packages/astropy/units/decorators.py", line 313, in wrapper
    return_ = wrapped_function(*func_args, **func_kwargs)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/taylor/miniconda3/envs/spelunker/lib/python3.11/site-packages/astroplan/plots/finder.py", line 88, in plot_finder_image
    hdu = SkyView.get_images(position=position, coordinates=coordinates,
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
IndexError: list index out of range




 >> Some spelunker tests failed. Please check errors above.

Additionally, these tests should ideally be run automatically to ensure that they are run and to save you devs some work and headaches. GitHub offers automated workflows that can be run on GitHub servers in various circumstances (e.g. for every PR). You could look at this file I helped make for Eureka! for inspiration.

This issue was made as part of the review for openjournals/joss-reviews#6202

The optimizing_photometry.rst file ends up being nested under https://jwst-fgs-spelunker.readthedocs.io/en/latest/timeseries_observations.html which feels a very unusual spot for it since it is an API document rather than a tutorial like fgs-spelunker-and-tsos.rst. I would recommend just moving the documentation of optimizing_photometry to the API page.

This issue was made as part of the review for openjournals/joss-reviews#6202

Right now, spelunker downloads data that does not have exclusive rights.

• Implement data download given a MAST API
• Document how to do this
• Return error or message the user about program IDs with exclusive rights.

I'd recommend adding a little bit more about the exact applications and functionalities of this package in the paper itself so that people who see this on the arxiv have a bit more of an idea whether this could be useful for their science e.g.

• instead of 'many possible analyses relevant to existing science programs' include some examples,
• add a figure or two from the documentation about how this data looks like
• a description of the 2D Gaussian fit e.g. regarding to identifying tilt events.

This is of course up to you!

This issue was made as part of the review for openjournals/joss-reviews#6202

Below is some feedback on your docs/user/fgs-spelunker_quickstart.rst documentation page. I've made this a check list so that it's easy for me to cross things off after you've made some updates.

I'll also note that this notebook is particularly useful in explaining many of the detailed operations of the code. You likely want to make this more enticing for people to click and read rather than just calling it "Spelunker Guidestar Tools" which doesn't make it seem as useful as it is as a more in-depth tutorial. Putting this right below the Quickstart would likely also help convince new users to check it out - this far down the list, I thought it'd be some random extra utilities.

• "The Program IDs that can be loaded are limited to programs without an exclusive access period or are available publicly." Is this still the case when using a MAST API token? If so, that's a really unfortunate limitation since it prohibits people from using the tool for their own data until the data become public (after which time most people will have already published their data). If not, then the ability to use MAST API tokens should be pointed out here.
• Currently, ``spk.readfile()`` does not support the folling attributes: - ``fg_table``, - ``object_properties``. Is this a comment meant to be noted outside of the code environment, or is this a warning message that is being output by the code? (I presume the former)
• "Current supported mnemonics: SA_ZHGAUPST (high-gain antenna), INIS_FWMTRCURR (NIRISS Filter Wheel Motor Current)". What barriers are there to accessing additional (arbitrary) mnemonics? It'd be amazing if folks could request any random mnemonic and get a result (I know there'd be several that I'd like to experiment with), but I also understand that you don't want to always grab all mnemonics since that'd be very wasteful.
• https://jwst-fgs-spelunker.readthedocs.io/en/latest/_images/fgs-spelunker_quickstart_51_1.png Would it be possible to make this a GIF for the purposes of the readthedocs page so that web users can see what this animation looks like?

This issue was made as part of the review for openjournals/joss-reviews#6202

When trying to install Spelunker with Python 3.12.1, I got the following error:

> pip install spelunker
Collecting spelunker
  Downloading spelunker-1.1.0-py3-none-any.whl.metadata (5.0 kB)
Collecting numpy (from spelunker)
  Downloading numpy-1.26.3-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (61 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 61.2/61.2 kB 2.8 MB/s eta 0:00:00
Collecting scipy (from spelunker)
  Downloading scipy-1.12.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (60 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 60.4/60.4 kB 6.4 MB/s eta 0:00:00
Collecting pandas (from spelunker)
  Downloading pandas-2.2.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (19 kB)
Collecting jwst (from spelunker)
  Downloading jwst-1.12.5.tar.gz (18.5 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 18.5/18.5 MB 11.0 MB/s eta 0:00:00
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Collecting astroquery (from spelunker)
  Using cached astroquery-0.4.6-py3-none-any.whl (4.5 MB)
Collecting astropy (from spelunker)
  Downloading astropy-6.0.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (9.8 kB)
Collecting astroplan (from spelunker)
  Downloading astroplan-0.9.1.tar.gz (140 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 141.0/141.0 kB 9.7 MB/s eta 0:00:00
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Collecting matplotlib (from spelunker)
  Downloading matplotlib-3.8.2-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (5.8 kB)
INFO: pip is looking at multiple versions of spelunker to determine which version is compatible with other requirements. This could take a while.
Collecting spelunker
  Downloading spelunker-1.0.4-py3-none-any.whl.metadata (5.0 kB)
  Downloading spelunker-1.0.3-py3-none-any.whl.metadata (5.0 kB)
  Downloading spelunker-1.0.2-py3-none-any.whl.metadata (4.9 kB)
  Downloading spelunker-1.0.1-py3-none-any.whl.metadata (4.9 kB)
  Downloading spelunker-1.0.0-py3-none-any.whl.metadata (4.8 kB)
  Downloading spelunker-0.5.5-py3-none-any.whl.metadata (4.4 kB)
  Downloading spelunker-0.5.4-py3-none-any.whl.metadata (4.3 kB)
INFO: pip is still looking at multiple versions of spelunker to determine which version is compatible with other requirements. This could take a while.
ERROR: Cannot install spelunker==0.5.4, spelunker==0.5.5, spelunker==1.0.0, spelunker==1.0.1, spelunker==1.0.2, spelunker==1.0.3, spelunker==1.0.4 and spelunker==1.1.0 because these package versions have conflicting dependencies.

The conflict is caused by:
    spelunker 1.1.0 depends on ray
    spelunker 1.0.4 depends on ray
    spelunker 1.0.3 depends on ray
    spelunker 1.0.2 depends on ray
    spelunker 1.0.1 depends on ray
    spelunker 1.0.0 depends on ray
    spelunker 0.5.5 depends on ray
    spelunker 0.5.4 depends on ray

To fix this you could try to:
1. loosen the range of package versions you've specified
2. remove package versions to allow pip attempt to solve the dependency conflict

ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/topics/dependency-resolution/#dealing-with-dependency-conflicts

It appears your ray dependency is incompatible with Python 3.12.1. However, when I downgraded to Python 3.11.0, the package installed successfully. I recommend you note this upper-limit on the Python version in your documentation and potentially in your setup.py file.

This issue was made as part of the review for openjournals/joss-reviews#6202

There are numerous currently unused arguments to the readfile function, specifically all of

visit_group='None', parallel_sequence_id='None',  activity_number='None', exposure_number='None', dir_seg='None', guider='None'

are not currently used in the function. It seems these arguments were previously used but the relevant code has currently been commented out. Most/all of these arguments are used in the download function, so is there any reason they're no longer used in the readfile function while the arguments are still available? Unless there is a clear reason to keep things as they currently are, I recommend you either enable the use of these arguments or remove them from the readfile arguments list.

This issue was made as part of the review for openjournals/joss-reviews#6202

Hi @GalagaBits,

Here's an issue I recently found with a dataset, on which the timestamps reverse in time periodically on the spelunker products. If you get data for PID 1366, observation 3, visit 1:

spk = spelunker.load(pid=1366, obs_num='3', visit='1', save=True)

And then you plot the time-stamps, you see something like this:

plt.plot(spk.fg_time,'.-')

So it seems the timing goes "back in time" I guess after each segment starts. It could be this is an issue with this particular dataset, but something to look at independently in general.

N.

The src/dashboard.py file appears to be entirely commented out. Do you intend for it to still be present, and if so why is it commented out? You also have a temp.rst file which I presume is no longer needed (given the separate docs folder, and given the "temp" name).

Additionally, you should not have __pycache__ or .DS_Store folders/files present in your repository (and the latter can actually be dangerous to store, from my understanding). Please add these to your .gitignore file and delete them from your repo.

Finally, I also recommend moving your JOSS paper files into a directory (calling it paper should still work with JOSS' automations I believe) to try to keep the main directory cleaner.

This issue was made as part of the review for openjournals/joss-reviews#6202

On the main JWST-FGS-Spelunker GitHub page, on the right panel beside About you should see a gear - if you click on that, then there should be a Website field where you can enter https://jwst-fgs-spelunker.readthedocs.io which will make it easier for future users to find the link to your readthedocs page.

This issue was made as part of the review for openjournals/joss-reviews#6202

In my first attempt to run your code, I got this error:

In [2]: import spelunker
   ...:
   ...: spk = spelunker.load(pid=2221)
jwstuser is not installed. mnemonics() will not work.
Traceback (most recent call last):
  File "<ipython-input-2-460cda62bfbc>", line 3, in <module>
    spk = spelunker.load(pid=2221)
TypeError: __init__() missing 1 required positional argument: 'directory'

The documentation doesn't include directory= as an argument, but it appears to be required.

As required by JOSS, there should be

clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

This can cleanly be done by completing the Code of conduct section at https://github.com/GalagaBits/JWST-FGS-Spelunker/community. JOSS' own code of conduct may be a good starting point: https://github.com/openjournals/joss/blob/main/CODE_OF_CONDUCT.md. The statement of

The pipline is currently under development on GitHub. For any bugs or requests, send us an email or open an issue on GitHub.

on your readthedocs page may suffice for JOSS' purposes, but adding something like that to your README would likely be a good step and adding a code of conduct would be ideal (in my opinion).

This issue was made as part of the review for openjournals/joss-reviews#6202

When trying to run all of the example notebooks to test the functionality of the package, I encountered a crash when trying to run the last cell of the pixel_centroid_mnemonics.ipynb notebook and was also not able to reproduce the figure output in the second-to-last cell.

For the second-to-last cell, I get this output image:

while the readthedocs page shows:

Meanwhile, for the last cell, I get a single figure that is successfully output:

although it looks very different from the readthedocs image:

but then I get the following error message:

UFuncTypeError                            Traceback (most recent call last)
Cell In[13], line 2
      1 filename = './event1_2079_xmean.gif'
----> 2 spk.flux_spatial_timelapse_animation(spk.fg_array[52000:53000], spk.fg_flux[52000:53000], filename=filename)

File ~/miniconda3/envs/spelunker2/lib/python3.11/site-packages/spelunker/spelunker.py:2232, in load.flux_spatial_timelapse_animation(self, fg_array, fg_time, fg_flux, start, stop, interval, filename)
   2229     im = ax1.imshow(short_fg_array[idx], animated=True)
   2230     im.set_clim(vmin=min, vmax=max)
-> 2232     im3 = ax2.vlines(i, np.min(short_fg_flux), np.max(short_fg_flux),  animated=True, color='red')
   2233     ims.append([im, im2, im3])
   2235 fig.suptitle('Guidestar spatial timeseries animation')

File ~/miniconda3/envs/spelunker2/lib/python3.11/site-packages/numpy/core/fromnumeric.py:2953, in min(a, axis, out, keepdims, initial, where)
   2836 @array_function_dispatch(_min_dispatcher)
   2837 def min(a, axis=None, out=None, keepdims=np._NoValue, initial=np._NoValue,
   2838         where=np._NoValue):
   2839     """
   2840     Return the minimum of an array or minimum along an axis.
   2841 
   (...)
   2951     6
   2952     """
-> 2953     return _wrapreduction(a, np.minimum, 'min', axis, None, out,
   2954                           keepdims=keepdims, initial=initial, where=where)

File ~/miniconda3/envs/spelunker2/lib/python3.11/site-packages/numpy/core/fromnumeric.py:88, in _wrapreduction(obj, ufunc, method, axis, dtype, out, **kwargs)
     85         else:
     86             return reduction(axis=axis, out=out, **passkwargs)
---> 88 return ufunc.reduce(obj, axis, dtype, out, **passkwargs)

UFuncTypeError: ufunc 'minimum' did not contain a loop with signature matching types (dtype('<U4'), dtype('<U4')) -> None

Also, note that the version of the notebook on GitHub is not exactly consistent with that on readthedocs, and note that the pid2079_observation04_visit1.dat file isn't uploaded to GitHub which requires that users run the spk.gauss2d_fit function which isn't described in the notebook.

This issue was made as part of the review for openjournals/joss-reviews#6202

Recently, some changes has been made to the parameters to the Guidestar Catalog from https://gsss.stsci.edu/webservices/vo/CatalogSearch.aspx. This is currently breaking Spelunker. I will release an update soon to fix this issue.

This linked code in the README.md does not exactly produce the subsequent figure shown in the file. I strongly recommend you adjust the code to include everything needed to reproduce the figure (including any custom rcParams settings) so that new users have a quick and easy way to validate their installation with this realistic use case.

This issue was made as part of the review for openjournals/joss-reviews#6202