galaxy ellipticity distribution test about descqa HOT 39 CLOSED

I'm on it, I can do that right now. @rmandelb any suggestions of what data to use as a reference ? It will depend on the shape measurement... I can pull ellipticity distributions from DES (for ngmix and im3shape) or the public HSC data

from descqa.

@evevkovacs @yymao Is it me or there is no columns related to ellipticity in the quantities exposed by the protoDC2 catalog reader ? I assume CatSim needs access to this information as well and I can see that in the native quantities there is some morphology information, but how these fields transform into an observed ellipticity if there is disk+bulge is not obvious....

I also see that the DESC shema defines a galaxy ellipticity, how is that one obtained ?

from descqa.

@EiffL good point. Can you create an issue at https://github.com/LSSTDESC/gcr-catalogs ?

from descqa.

sure thing

from descqa.

Sorry, I forgot to add a reply. Ellipticity is available in the native quantities. See morphology/diskEllipticity
and morphology/spheroidEllipticity. The definition will be changing in the next version of the catalog (coming soon) and more documentation will be provided in the accompanying new version of the protoDC2 note.

from descqa.

@jablazek @rmandelb @EiffL @evevkovacs @dkorytov @patricialarsen Just to loop everyone in on this issue of how to implement a validation test on galaxy ellipticity distribution with two-component shapes and radial profiles --- I think the conclusion today is to do some kind of weighted sum to generate total shapes. What's the weighting scheme we will be using? I assume we can do that at the reader level? Any other thoughts on this?

from descqa.

@jablazek @rmandelb @EiffL @dkorytov @patricialarsen @yymao The current plan is to use a luminosity-weighted average of the magnitude of the ellipticities for disk and the spheroid. Rest-frame luminosity in either r or i will be used. e1 and e2 will then be computed from the position angle.
It could be done at the reader level, but Dan has to fix a bug in v2.1, so he's just going to add this quantity to v2.11, and then expose it in the reader.

from descqa.

@evevkovacs @dkorytov Wouldn't it be better to still do it at the reader level so that we can more easily try out different weighting schemes/ratios?

from descqa.

All the needed quantities will be available so additional models can be added via the reader as desired.

from descqa.

@evevkovacs @dkorytov I'm trying out another way of computing the total ellipticity by adding the second moments, but I'm running into 2 issues with proto-dc2_v2.1.1

• It looks like ellipticity_1 = ellipticity_2 for all objects, which is a little bit suspicious...
• What is the unit of morphology/positionAngle ? It doesnt look like radians or degrees

from descqa.

from descqa.

from descqa.

perfect, thanks !

from descqa.

@dkorytov Hum.... sorry, the ellipticity still looks really weird...
Here is the distribution of ellipticity_2

it's suspiciously only positive and very close to 0.

The distribution of morphology/diskEllipticity2 looks ok:

But I'm very surprised at the distribution of morphology/spheroidEllipticity2:

Is that expected ???

from descqa.

from descqa.

@EiffL and @evevkovacs, any updates on this test?

from descqa.

@yymao - it looks like as of the last discussions we were waiting to hear from @dkorytov about the strange things Francois was finding.

from descqa.

@rmandelb sorry --- I should have added more detail in my comment. On the last day of the Sprint Week, @dkorytov has updated the protoDC2 catalog to fix the ellipticity issue (see LSSTDESC/gcr-catalogs#48). So I am now following up to see if the fix actually works.

from descqa.

@yymao Let me have a quick look

from descqa.

Nope, I think Dan made some update but not after I made these plots last week. Here is the e2 component in the latest version of the reader:

from descqa.

@EiffL thanks!
@dkorytov and @evevkovacs, can you take a look at this issue?

from descqa.

@yymao I will ping Dan who is in Mexico and take a look myself. I have the test code partially done.

from descqa.

Hmm...I thought I fixed that...

I'm changing the ellipticity model to copy data from galaxy zoo (https://arxiv.org/abs/1306.3264, fig 4) The current geometric model has a spike in the distribution while the data doesn't.

from descqa.

I just ran a notebook to plot ellipticity1 and ellipticity2. Here's my code:
gc = GCRCatalogs.load_catalog('protoDC2')
edata = gc.get_quantities(['ellipticity_1', 'ellipticity_2'])
fig, ax = plt.subplots(1,2)
ax[0].hist(edata['ellipticity_1'])
ax[1].hist(edata['ellipticity_2'])
and here's the result

from descqa.

@evevkovacs @EiffL thanks --- I think your plots actually agree with each other --- very narrow spiky distribution near 0.

@dkorytov mentioned that he's changing the ellipticity model --- is that something in progress or already implemented in v2.1.1?

from descqa.

@yymao @dkorytov Well there's some confusion about which component is off. Dan is looking into it. Apart from changing the model, there is also a bug which should be fixed in v2.1.1, regardless of whether we produce a new minor version.

from descqa.

@EiffL @rmandelb (cc to @yymao ) As discussed, I will start working on integrating plots of the total ellipticities and the components into DESCQA. Can you point me to some sample validation data? It doesn't have to be the final data set, but I'd like to get an idea of what selection cuts I need to be making on the catalog data. For example, will I need to make a selection on the redshift range and/or on the magnitude of the galaxies included? Are there any other selection cuts I should include? It's better to construct the test with those options setup from the get-go. Thanks

from descqa.

Maybe @rmandelb will point to a more recent study, but this makes me think of the study that was done in this paper: Joachimi et al. 2013 where they used COSMOS galaxies and look at trends with redshifts and magnitude cuts, as well as for different galaxy types.

from descqa.

That's a good idea. I was struggling a bit with this because most of the samples I can think of have some hard-to-reproduce cuts, or don't divide by the quantities we are interested in, or only report an RMS ellipticity but not a distribution. COSMOS is probably the way to go, and Benjamin Joachimi could presumably give us the distributions in figure 7 of that paper. In the meantime, the caption and surrounding text hopefully gives Eve a basic idea of how to cut up the samples to compare with the data.

from descqa.

Paging Benjamin @joachimi : we are interested in getting a machine-readable version of the data in figure 7 of your 2013 paper to compare against our simulations; can you please help us out?

Many thanks!

from descqa.

Thanks, that would be great. I started looking at the paper. from Fig 5 I infer that the redshift range is 0-1.5, but wasn't sure that was true for all the plots. The other question I had was regarding the various morphology cuts. How are selections for early, late, disk dominated and bulge dominated defined? I didn't see definitions in the paper, but maybe I missed them. And what's the closest equivalent to V band in LSST/DES filters? (protoDC2 has V band but buzzard does not)

from descqa.

V band is sort of midway between g and r. I would suggest picking one of those and not worrying about it too much.

Section 4.1 describes the morphological classifications that lead to a definition of early-type, bulge-dominated late type, disk-dominated late type, and irregular. They are from an image-based algorithm that we cannot really reproduce here, but since this is a sim you should just use the ground truth. For example, you could define bulge-to-total ratio = BTT = (bulge flux) / (total flux) for each galaxy, and use those with BTT > 0.7 as early type, BTT < 0.2 as disk-dominated late type, 0.4<BTT<0.7 as bulge-dominated late type. Obviously this is not an exact match to the data but it's reasonably well-motivated.

Note that they also have a cut on F814W<24 (section 4.1). This is basically i<24 (modulo tiny band offsets). So you should impose that cut for this shape comparison as well.

By the way, their shape definition appears to be the one with (1-q^2)/(1+q^2), q=b/a.

Does that make sense?

from descqa.

@rmandelb Thanks very much, that is a big help. I noticed the F814W cut, but forgot to ask for the LSST equivalent. Thanks for picking up on that. We switched the ellipticity definition to (1-q)/(1+q), as per your suggestion some time ago, but the test can figure out the change to the above definition.

from descqa.

Yeah, sorry. The advantage of (1-q)/(1+q) is that you can take the ensemble average or whatever and get the shear (whereas with the other definition, the ensemble average is not equal to the shear). So the definition I recommended earlier is useful for lensing tests. It's just not great for comparing with this paper :)

from descqa.

@evevkovacs - as I understand it from our discussion offline, you are working on ingesting the suggested validation data for p(e) from Benjamin (thanks!) so it can be incorporated into a validation test.

Does this mean that what you primarily need is a validation criterion from @msimet (on behalf of WL) or do you also need some additional contributors to the code base (e.g. to implement part of the test / look it over)? I know that @EiffL has been involved in this as well.

from descqa.

@rmandelb Sorry this one got lost in my email stack. Yes, I mostly need validation criteria. I'll try to have some example distributions by next Thursday, but have been working on the new protoDC2 as well, which is also time critical.

from descqa.

Thanks for clarifying, Eve. I am sure the weak lensers can provide some validation criteria; it seems likely we would just want to have rough tests of percentile values (i.e., the |e| values for say the 16th, 50th, and 94th percentile) so that we know the distributions are sensible. However, it may be best if we can see what the plots look like before deciding on the exact form of the test.

I completely understand this might be lower priority than the new protoDC2.

from descqa.

@evevkovacs just want to check in to see if you need help on this test. I understand that you have been working on many things right now so I'd like to share your workload if possible. If it makes sense, I can find someone (or find myself) to implement this test since it is relatively straightforward. Let me know what's the current status.

from descqa.

Done in #113

from descqa.