sphemakh / fidelity Goto Github PK

I've been thinking of how best to estimate the noise in an image, because simply taking the std dev or mad (median absolute dev) of the pixels does not give accurate results in the presence of bright emission. But, since the sky emission is positive, the negative pixels should give a far better estimate of the noise. This is what I'm thinking:

Let D be a N_x_N image and d be the negative pixels of D (1x_M_, where M is the number of negative pixels).
Then, assuming that the noise is symmetric about zero the noise can be approximated as:
sigma = std_dev[d || (-d)], where || represents concatenation.

In the presence of calibration artefacts (which generally have negative counterparts) this might be a bit off, but it should still be an improvement over using all the pixels.

The variance of an image should vary with in an image due to beam response, we should compute a local variance and then look for structures in the residual image which diverge from that variance, these are probably artefacts.

An ideal residual image will have Gaussian noise (with some bias towards positive values be cause sources have positive flux). We can fit multiple (or general) distributions to the histogram of the residual image pixel values. Then ask which distribution best models the residual noise, and if that is close to Gaussian.

It would be useful to try general Gaussian, Lapacian, Cauchy distributions. And compute higher order moments such as skew and kurtosis.

Just to get us going:

I've add a script that does some of the things we've talking about:

• It fits a distribution (cauchy, gaussian, laplace, maxwell) to the image pixels and returns the best fit parameters with their errors.
• Calculates the "local variance" at source locations (this requires a Tigger compatible sky model)

This is an example of how to run the script:

image_stats/stats.py -fit laplace -s -nb 200 -cat nvss6deg.lsm.html foreman_grill_0_s1-briggs-noise.fits

The artefact structure in a residual image should be 'similar' to that of the PSF sidelobes. Given a PSF image, remove the main lobe, to get only the sidelobe structure, does this have a correlation with the residual image?

Also, perhaps the image should be downsampled (before or after?) correlation to improve correlation?