Dear @kbelisar
Thank you for making this package. These plots are very eye-catching. I like it.
But let me, please, kindly ask you to consider changing the SEs to either SD (for description) or CIs (for inference). SEs itself has no direct application in statistical inference, as no distribution of the test statistic is employed.
SE, measuring the precision of sampling, itself is just a semi-product, used to calculate the confidence intervals depending on the used distribution. The SE itself says nothing about the comparison as long as we don't pick the right distribution - and this distribution doesn't have to be even symmetric, like chi2, F, non-central T.
As a consequence - for the same SE we may have (very) different CIs, depending on the used test statistic. This may result in different result of testing for the same SE.
Unfortunately to science, researchers:
-
routinely naively look for overlap of SEs to compare groups, which leads to wrong conclusions. SEs are just too short, so the lack of overlap doesn't necessarily mean statistical significance. Actually, comparing group CIs is also wrong (wrong SD is used) - oppositely they are too long so overlapping doesn't necessarily mean lack of stat. significance. The only CI that makes sense for the inference on comparing groups is the CI of the difference or ratio.
-
even worse - they often "abuse" SEs by making use of the fact, that SEs are shorter than CIs (SE is "expanded" to a CI through the quantile of the test statistic distribution ), which is used to "show up" the effect and cheat that it is "better visible".
Given the above arguments, would you be willing to change SE to CI?
In the simplest case - by assuming the t-distribution.
Ideally - flexibly: by allowing the user to specify the distribution of interest or BCa bootstrap, by returning the by default the BCa (bias corrected and accelerated) CIs, which, if failed, are calculated as percentile CIs.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent