kinto-b / elections Goto Github PK

In c618bbe I increased the variance on the priors for what is effectively the division level swing and on the coefficients of the demographic covariates. This is good insofar as it lets the record-level data have more influence on the estimates. Prior to the change the estimates were anchored really strongly to the previous election result, which wasn't ideal.

However, this introduced pretty serious computational issues. The (cleaned up) stan output reads:

1: Divergent Transitions Warning:
There were 1788 divergent transitions after warmup. To understand why this is an issue and how to address it, visit:
https://mc-stan.org/misc/warnings.html#divergent-transitions-after-warmup
(Package 'StanHeaders' was built under R version 4.2.3)

2: Sampling Problem Diagnosis:
Examine the pairs() plot to identify potential sampling issues.
(Package 'StanHeaders' was built under R version 4.2.3)

3: Low Bulk Effective Samples Size (ESS) Warning:
Bulk Effective Samples Size (ESS) is too low, indicating potential unreliability of posterior means and medians.
Running the chains for more iterations might help.
(Package 'StanHeaders' was built under R version 4.2.3)

4: Low Tail Effective Samples Size (ESS) Warning:
Tail Effective Samples Size (ESS) is too low, suggesting potential unreliability of posterior variances and tail quantiles.
Running the chains for more iterations might help.
(Package 'StanHeaders' was built under R version 4.2.3)

Divergences:
1788 out of 10000 iterations encountered divergences (17.88%).
Consider increasing the 'adapt_delta' parameter to address the divergences.

Tree Depth:
None of the 10000 iterations reached the maximum tree depth of 10.

Energy:
E-BFMI indicates no pathological behavior.

The large number of divergent transitions is particularly concerning.

Generalise the code to accept data from different years, then run the pipeline using data for the 2022 election. This will also set us up to run a 2025 prediction.