egouldo / manyanalysts Goto Github PK
View Code? Open in Web Editor NEWSource code for ManyEcoEvo Manuscript
Home Page: https://egouldo.github.io/ManyAnalysts/
License: GNU General Public License v3.0
Source code for ManyEcoEvo Manuscript
Home Page: https://egouldo.github.io/ManyAnalysts/
License: GNU General Public License v3.0
SM3: Table C5, change 'mixed model' to 'Random Included' or some version, so that is consistent with other language throughout.
Line 938 I think a formula for
$Zr$ is warranted here.
and/or add citation. I think I struggled to find this in the literature but ended up finding it in Mengersen's meta-analysis for ecology handbook back when I double checked the calculation originally when coding the analysis.
Chunk descriptive-munging is no longer executing, giving the following error on knit:
processing file: index.qmd
|.... | 11% [descriptive-munging] Error in `UseMethod()`:
! no applicable method for 'inner_join' applied to an object of class "character"
Backtrace:
1. dplyr::inner_join(BTSorensen, effect_Ids)
Quitting from lines 2418-2789 [descriptive-munging] (index.qmd)
Execution halted
Also, I note that this code is duplicated in SM2_EffectSizeAnalysis.qmd, however in the latter document, the objects are mostly subset.
Currently only can navigate to Appendices but would be better to also navigate around main text as is super long.
- The data and code to reproduce the results in the manuscript is provided in their R package (hosted at Zenodo and GitHub account of first author) called
ManyEcoEvo
.
- provide the link (https://github.com/egouldo/ManyAnalysts/tree/main) (not just to the rendered HTML manuscript at https://egouldo.github.io/ManyAnalysts/, which I note is in the supplementary file)
- cite the software they used for writing their manuscript in their references, i.e. Quarto, gt and ggplot2
- The formatting in HTML manuscript was more pleasurable and it would have been nice to have been alluded to this earlier rather than discovering it later
Each additional post-hoc analysis that is included nullifies every positional reference in in-text R expressions. After updating for #27 will need to ensure that EVERY single in-text expression is matched semantically (by column and row values) rather than by position.
Next, we will assess the relationship between the inclusion of random effects in the analysis and the deviation from the mean effect size. We anticipate that most analysts will use random effects in a mixed model framework, but if we are wrong, we want to evaluate the differences in outcomes when using random effects versus not using random effects. Thus if there are at least 5 analyses that do and 5 analyses that do not include random effects, we will add a binary predictor variable random effects included (yes/no) to our set of univariate analyses and will add this predictor variable to our multivariate model described below.
Thanks Tim, I think I was just relying on the two headers being under their respective sub-headers for the precise meaning of each sub-heading being conveyed. But it’s probably best to specify exactly what outputs these belong to, so I’ll update them.
Elliot.
On 1 Jun 2024, at 6:34 am, Tim Parker [email protected] wrote:
Note this redundancy between these two supplement sections
A.2.3 Post-hoc analysis: Exploring the effect of removing analyses with poor peer-review ratings on heterogeneity
B.1.1.2 Post-hoc analysis: Exploring the effect of removing analyses with poor peer-review ratings on heterogeneity
Line 988 should the “meta-analytic mean Zr or yi” have bars over them? I think so, to indicate mean value.
I don't think the following statement is correct:
The blue tit dataset had a total of r length(unique(var_names[[1,2]][[1]])) candidate variables, each of which were used in at least one analysis.
Based on what I observed when removing bugs in descriptive munging chunk to incorporate collinearity analyses.
Line 2141 in 28c4a40
For some reason, in the rendered version, two of Shinichi's citations are including his first name in the rendered text. In this line, it appears as:
linear models [GLMs; Shinichi Nakagawa and Cuthill (2007).
Shinichi's 2022 paper also has this problem. I have looked that the citations in the .bib file, and I cannot see any difference in formatting compared to other citations. I have not seen this problem arise for any other citations.
I didn't quite understand Supp A.1.3 where it said that XX candidate variables were used in a mean of YY Zr analyses. Does that mean YY analyses used all XX candidate variables?
SD misleading (larger than mean) if don't know that distributions are skewed. We will add ranges for all descriptive statistics.
Note, these are already calculated in object Table2
as min
and max
.
Hey Tim,
I think this may potentially be a result of the way I repeated the analyses over each of the different data subsets, i'll look into it.
So, the models may have been fitted automatically, but I never extracted the results of the cases where we had < 5 analyses with random effects included.
Elliot.
On 1 Jun 2024, at 6:14 am, Tim Parker [email protected] wrote:
Hi Elliot,
In the supplement
C.6.1 Effect Sizes Zr
also
C.6.2 Out of sample predictions yi
It appears that the presence of mixed effects (whether or not the analyst included a random effect) was included in both the BT and the Euc models, but I thought that we only included this effect for Euc since there were too few BT analysts that did not include a random effect.
-Tim
Where did this figure come from??
r All_Teams+28
call in Summary Statistics section
I included a placeholder citation for the new forest plot that excludes the effects from blue tit analyses containing the two highly collinear predictors.
This placeholder is "(@fig-forest-plot-?????)"
The table captions (but not the figure captions) are centered instead of left justified. Also, there narrow lines above and below most of the specially formatted text.
Page 45, Line 1005-1006 suggest that the ReviewerID and AnalysisID have the same variance. I think the authors meant this to be different so
$\sigma^2$ should have different notations. Same for Page 49, Line 1043-1044.
For the univariate analysis of the Deviation Score, I don't see mention of weighting based on the inverse variance however looking at the analysis, it looks like the authors did this for continuous rating (https://github.com/egouldo/ManyEcoEvo/blob/main/R/fit_boxcox_ratings_cont.R#L44) but it's commented out for categorical rating (https://github.com/egouldo/ManyEcoEvo/blob/main/R/fit_boxcox_ratings_cat.R#L52). I'm wondering why?
Can't recall exactly why, and maybe this is an error, but from memory I think there were some problems with model fit for the categorical ratings and inverse variance?
Model not included in that parameter table.
Check that model parameter estimates for Sorensen's models are included somewhere in that document.
In the Supp Mat (SM3_ExplainingDeviation.qmd), Subsection C.4.1 Effect Sizes (deviation scores as explained by distinctiveness of variables in each analysis) is empty...?
Is this because this content occurs elsewhere in the manuscript?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.