X-ray: Generate and Analyse is a module designed to make the analysis of XMM observations simple and efficient. It provides an interface with SAS for the creation of XMM data products, as well as a way to easily perform fits (scalable for multiple observations) and retrieve information about an object, all within a Python package.
Not really an immediate issue, but there should be some checking for resolutions other than the XCS/XGA standard of 512x512, otherwise things like PSFgen are going to get really confused. - If there are not XCS resolution then they should probably only be used to read in region files.
I made the design choice to use a search aperture of radius 500kpc for the iterative peak finding algorithm, but of course that proper radius can't be converted to degrees and then to pixels without redshift information.
Not difficult to fix, I'll probably just add an if statement that says if there is no redshift use a search aperture of 5 arcminutes or something.
The structure of the module could be made a little better, in terms of the internal file structure. I've already split up the product classes into more sensible files and chunks, I should probably do the same for the rest of the project.
The GC class needs to take common cluster observable measurements as arguments (such as richness, Y, WL mass etc.)
This will make the cluster source objects more useful when it comes to making scaling relations, as well as giving some multi-wavelength context to the X-ray observables.
Sanders et al papers from the early 2000s had some really nice binning methods for ratemaps in them.
Could be really good to implement them in XGA, because the regions could then be patched in to the spectrum generating functionality to measure more localised temperature maps.
Also binning like that could allow me to come up with a better deprojection method that takes into account localised temperature and density changes better.
For the GalaxyCluster source object to be useful, it needs to require the passing of overdensity radii. One of the simplest ways we characterise the extent of a cluster.
It then needs to setup regions with those radii, and centre them either on the user-passed coordinate, or the X-ray centroid.
Was exploring off axis clusters using the XCS-DES sample Reese provided, found a very bright off axis cluster that is ideal for my needs, his table reports it having XMM coordinates of:
RA: 313.98311792357896
DEC: -54.9302122469135
Which quite clearly lines up with a cluster and is solidly within this observation (0675010901).
However, when I define an XGA source it cannot find a matching source for some reason.
Many global X-ray measurements of clusters are performed on 'core-excised' spectra, so that should be an option in XGA as well. A core-excised flag should be included in the file name - fitting and results storage mechanisms will need to be adjusted to be able to store fits to core-excised and non-core excised spectra of the same source
I don't currently generate overdensity radii masks/ custom masks for individual observations and instruments, so get_masks shouldn't successfully return anything in this case.
If a nuisance point source is detected as interloping in a cluster, it won't be removed from a background region. So if a point source is right on the edge of a region and present in both the source and background regions, it won't be properly removed.
For instance, defining a source with name="XMMXCSJ142601.0+374937.0", results in a SAS error:
xga.exceptions.SASGenerationError: datasetCouldNotBeRead raised by rmfgen - The dataset with name '0112230201_mos1_XMMXCSJ142601.0' could not be read; either it does not exist or it is in an unrecognised format
The dataset name reported in the error is not the full spectrum path, its cut off just before a "+" symbol, suggesting some SAS procedure has an issue with that particular symbol.
The error does not seem to occur when the "+" is replaced by a "-"
For instance, if a method wants an angle argument, and the calculation expects it to be degrees, some methods (like _setup_new_region), will fail if the argument is not in degrees, even though an argument in arcseconds would be totally fine.
Feel like I should attempt to calculate the amount of a given source that lies on a detector, raise flags if its too low, if the user coordinates lie in a chip gap etc.
On declaration of a source, XGA can hunt round for products it generated previously. I need to expand it to support PSFs, and eventually the deconvolved image files I will make.
Not for everything, just for the key or 'new' bits. The motivation of this is mainly because this is a science module, and one of the major points of making this code public and installable is so it can be understood and appraised/criticised by anyone in the community.
For instance I should do one on the hierarchical clustering peak finding method I came up with.
Also for the PSF deconvolution stuff I'm planning (wrt the rebinning etc), assuming this works at all and makes it into the module.
Although the stacking functionality remains, I don't think it really needs to. If it were removed it would clean up sas.py slightly, as well as some of the BaseSource class.
This is a planned feature which is essential for making this module viable for large scale analyses. Doesn't matter on the short term, but certainly should be done for beta release.
What I suspect will happen is I'll have to look out for the correct log files to know when a job has finished - then I can read the log files in as stdout and stderr and carry on much like I do already.
I'm sure this is probably what you had to start off with, but it achieves what you want with the creation of as few extra variables as possible (so we don't end up with unwanted references).
It's so frustrating that you can't reliably recreate this error...
Currently, a custom region will just save with a 'custom' flag in the spectrum filename, but it perhaps it should depend on where the custom region is so that multiple custom regions can be saved and analysed?
The hierarchical clustering peak finding method seems to work very well, but I should return the point clusters from the method and store them as attributes of the source class.
That way they can be used to make plots to check the peak finding has worked well.
When an XSPEC fit is run from a Jupyter Notebook, something funny goes on with the creation of the summary fits file. The actual XSPEC run and fit complete fine, but the script fails when it comes to write out the results.
Errors like this get thrown:
Unable to redirect prompts to the /dev/tty (at headas_stdio.c:152)
ERROR: No such device or address
Task ftcreate 0.0 terminating with status 6
The same fits work fine when run outside of Jupyter Notebook
From the ratemaps and masks construct radial brightness profiles.
Ideally I'd like to be able to do this with a dependence on angle as well, and scrap some of the assumption of spherical symmetry, but initially I think just radial profiles will be fine.
I tried to install XGA v0.1.0 using pip on the workstation and I think the version requirements were causing some issues. Either way I should simplify them to greater than equal statements were possible. Regions has to remain at v0.4.
This will also need to change in the requirements.txt.
Also, as something also in setup.py that isn't worth of its own issue, I need to add a long description to setup.py so it appears on PyPi on the next release.
For the ratemaps to be of any real use in proper analysis (such as measuring accurate gas masses), I'm going to have to deal with the background. The simplest solution is simply to define a background annulus (which I have done), average it and subtract from the ratemap - this is probably what should be implemented first.
Otherwise, I know it is possible to generate some more sophisticated background subtracted images with eSAS, so maybe I'll look more into that.
This seems to be an oversight on my part. When the ExtendedSource init is called with super from GalaxyCluster init, it checks to see if there is a custom region, if there is no custom region and no matching region files then it can't continue because it doesn't know about the overdensity radii required by a GalaxyCluster object.