theupshot / leo-senate-model Goto Github PK

Would be great to get a daily dump of the poll data. Otherwise it's hard to replicate the NYT's own math, follow along at home or explore alternate assumptions.

The senate-likelihood.tsv seems to end up with strange results and a huge run-to-run variation. In four successive runs of Rscript master-public.R, I got the below. It doesn't seem like this could be correctly reporting the probability estimate of a Democratic-controlled senate on those dates -- the variation between runs is too big (between 35% and 50% chance of retention?).

date dem
1-Jan-14 60
25-Feb-14 55
22-Apr-14 35

date dem
1-Jan-14 40
25-Feb-14 40
22-Apr-14 50

date dem
1-Jan-14 75
25-Feb-14 40
22-Apr-14 45

date dem
1-Jan-14 85
25-Feb-14 70
22-Apr-14 35

I'm imagining this is mainly an individualized problem, since I don't see any other error postings on here -- but if it's not, here's a screenshot of my output when the simulation starts:

Just as a reality check, how long does it take you to run this with n.sims <- 250000, and what kind of system are you running it on?

For us it's been running for about two hours on a 4.4 GHz Core i7-2600K, saturating one core, and the numbers counting down (after "SIMULATING THINGS") have gone from 307 down to 220. It's consuming about 9-12 GiB of RAM.

Do you run it on lots of EC2 nodes and have some technique to combine multiple separate runs (with a smaller n.sims in each run) or otherwise multithread it? At this rate, running on one core it looks like it will take more than a day to finish. :-)

Thanks,
Keith

It might be a good idea to fix the random seed used for the 250,000 realizations of the simulation, in case readers are interested in reproducing exactly the numbers reported on the NYT Upshot Web site.

I have tried with today's data and cannot quite match the figures on the Web site. For example, even after trying several times, I can't get up to >88.5% for New Hampshire. (The public figure is 89.)

But there is some run-to-run slop that comes from the randomness, and it would be easier to debug if I knew I was using the same random seed as the public figures.

I was interested in using this code for a project, but as it stands, I'm stumped on how to get it up and running. When I run the basic code, this is the output I get. Does anyone have any suggestions?

1. Thanks for this
2. Would it be worthwhile to just include the published output? Unless the code is going to be changing on a frequent continuous basis? At the least, a sample inventory can be posted (I'll submit a pull request in just a bit)