Test out Gaussian Process for dealing with outliers about assaytools HOT 8 OPEN

I don't think this is what we want. GPs are great for data in which there is a natural spatial relationship between the collected data, but that relationship must be learned. We are dealing with a very different case---we know what the relationship is, through the dissociation constant equations and mass conservation laws. Utilizing a GP of the sort in those examples would not only "forget" that information, but it doesn't allow us to propagate any uncertainty in which points are outliers into the posterior.

from assaytools.

Instead, I think we should use an approach like this, where there is a prior on the fraction of outliers and the outlier distribution has a mean and variance that is inferred (and marginalized out) during MCMC sampling:
http://www.astroml.org/book_figures/chapter8/fig_outlier_rejection.html

from assaytools.

But first, before we even talk about models, we absolutely need to collect some examples of the outliers and look at them to see what it tells us about the nature of the data.

from assaytools.

Just making a note here that this is something we should keep at the front of our minds.

from assaytools.

Agreed! Would be great to compile a list of data with outliers to find a strategy that works!

from assaytools.

Here is an example of two almost identical datasets with and without an outlier:

With outlier:
https://github.com/choderalab/fluorescence-assay-manuscript/blob/fig_sketches/analysis/bayes/DMSO-backfill/delG_Bosutinib-AB-2016-07-31%2020:10.png
https://github.com/choderalab/fluorescence-assay-manuscript/blob/fig_sketches/analysis/bayes/DMSO-backfill/Bosutinib-AB-2016-07-31%2020:10.json

Without outlier:
https://github.com/choderalab/fluorescence-assay-manuscript/blob/fig_sketches/analysis/bayes/DMSO-backfill/delG_Bosutinib-IJ-2016-07-31%2020:13.png
https://github.com/choderalab/fluorescence-assay-manuscript/blob/fig_sketches/analysis/bayes/DMSO-backfill/Bosutinib-IJ-2016-07-31%2020:13.json

from assaytools.

Awesome! This is exactly what we need to make this work! Thanks!

from assaytools.

@jchodera has an idea about Bayesian outlier detection that he is interested in implementing.

from assaytools.