Comments (7)
I added some documentation to the function. p_emp
is the important item here, the other outputs can mostly be ignored. The function is simply a bootstrap test for testing whether the median score difference is different from zero. It's a bit awkward because the scores are log-likelihoods, but for this test all that matters is that the input vectors come from two models evaluated on bootstrap resampled SNP blocks (using the same resamplings for both models). You could also supply worst f-statistic residuals, or any other estimated model parameter as input.
p_emp
is two times the fraction of bootstrap replicates where model 1 has a lower score than model 2 (or vice-vera, whichever is less), truncated at 1/(number of bootstrap replicates).
ci_low
and ci_high
are the 2.5% and 97.5% quantiles of the score difference distribution. And since the input vectors represent bootstrap replicates, that should be the same as the 2.5% and 97.5% confidence intervals of the score difference distribution.
from admixtools.
from admixtools.
Yes, you can interpret it like a p-value, but it's a two-sided test, so it tests whether either of the two models is significantly better than the other model.
from admixtools.
from admixtools.
Sorry for not being clear!
Low values of p_emp
suggest that the difference in scores between the two models is unlikely to be due to the variability across SNP blocks. But this value will be low if model 1 has higher scores than model 2, or the other way round.
If you want to know which of the two models is better, you can look at whether the diff
value is positive or negative. That's the estimate of the mean difference scores1 - scores2
, so positive means model 1 has higher scores (is worse).
That difference will almost always have the same sign as the score difference that you get when evaluating both models the standard way (using all SNPs with qpgraph
, not using bootstrapping with qpgraph_resample_multi
). If you have already evaluated both models the standard way, you probably already know which of the two models has a lower score, and you just want to test whether that difference is significant. That's why I didn't emphasize the diff
output here!
from admixtools.
from admixtools.
Thanks for the question and answer, but I may need some clarification for some of my results. One of my model (Nadmix = 1) has a score of 0.438 and the other (Nadmix = 2) has a score of 0.000603. However, when I run the qpgraph_resample_multi and copmare_fits functions, I get the following:
$diff
[1] -3.223325
$se
[1,] 6.29504
$z
[1,] -0.5120419
$p
[1] 0.6086217
$p_emp
[1] 0.6
$p_emp_nocorr
[1] 0.6
$ci_low
[1] -22.78158
$ci_high
[1] 2.032134
This suggests to me that none of the models is better than the other. note that I also get a similar result (i.e. no significant p. val.) when comparing these models with higher admixture events (up to 7).
Is my interpretation correct?
Thanks in advance!
from admixtools.
Related Issues (20)
- Optimization unsuccessful (constraints inconsistent). Try increasing 'diag', when running qpgraph with a graph and upper and lower bounds HOT 2
- Error in parse_qpadm_output HOT 1
- The order of pops in outgroup f3 test HOT 2
- Explanation of the functions qpdstat and qpf4ratio HOT 1
- Protocol Advice: Establishing a t hresholds for the number of allowed admixture events HOT 3
- qp3pop unique_only does not work as expected HOT 2
- Do qpWave, qpAdm, and f5 have requirements on the minimum number of individuals for each population? HOT 1
- est_to_boo does not preserve the SNP block names (number of SNPs), but est_to_loo() does. Is that intended? HOT 2
- large difference of f3 result between admixtools and admixtools2 HOT 1
- Zero drift edges HOT 2
- Behavior affected by other R packages? HOT 2
- Error in if (gimp > 0 && gimp%%plusminus_generations == 0) { : missing value where TRUE/FALSE needed HOT 2
- please add tags/releases HOT 2
- Inconsistent bootstrap significance testing HOT 5
- keyword argument typo lack of error HOT 3
- Comparing graphs with compare_fits HOT 2
- auto_only: change default value to FALSE HOT 1
- qpWave returning one fewer ranks than expected HOT 1
- qpAdm computation stucks when the num. of letf pops below 3
- Issue with f3 and f4 HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from admixtools.