Light

newsdev / nyt-campfinbot Goto Github PK

View Code? Open in Web Editor NEW

8.0 28.0 2.0 29 KB

A SlackBot for notifying NYTimes reporters and editors about filings to the Federal Election Commission's web site.

Python 100.00%

fec slackbot reporting-tool

nyt-campfinbot's Introduction

A Python-based SlackBot derived from SlackHQ's remarkable RTMBot

Getting started

This bot uses MongoDB instead of the filesystem for persistence. You can install MongoDB locally (default) or export CAMPFINBOT_MONGO_URL with your custom connection string.

brew install mongodb
ln -sfv /usr/local/opt/mongodb/*.plist ~/Library/LaunchAgents
launchctl load ~/Library/LaunchAgents/homebrew.mxcl.mongodb.plist

Create a virtualenv and install the required packages.

mkvirtualenv nyt-campfinbot && pip install -r requirements.txt

Create a bot integration for your campfinbot and get the channel ID. Then export the channel ID and the token.

export CAMPFINBOT_SLACK_CHANNEL=C012345
export CAMPFINBOT_SLACK_TOKEN=ABCEFGHIJKLMNOPQRSTUVWXYZ01234567890

Export the hosts for preloaded data.

CAMPFINBOT_CANDIDATES_HOST=interactive-api.newsdev.nytimes.com
CAMPFINBOT_FILINGS_HOST=projects.nytimes.com

Create the log file if it doesn't exist.

touch /tmp/campfinbot.log

Preload data for the bot. It needs some committees and would appreciate knowing about old filings so it doesn't spam your slack channel with stuff you already know about.

python -m campfinbot.preload

Run the bot itself.

python -m campfinbot.bot

Tail the log to see what's going on.

tail -f /tmp/campfinbot.log

Deployment

Ubuntu Linux

Make an Upstart script in /etc/init/campfinbot.conf and use this template.

start on runlevel [2345]
stop on runlevel [!2345]

respawn

script
  export CAMPFINBOT_CANDIDATES_HOST='interactive-api.newsdev.nytimes.com'
  export CAMPFINBOT_SLACK_CHANNEL='C012345'
  export CAMPFINBOT_FILINGS_HOST='projects.nytimes.com'
  export CAMPFINBOT_SLACK_TOKEN='xoxb-1234567890-AbcDefGhijkLmNOpQRstUvWXyz'
  export CAMPFINBOT_PRD_HOST='ec2-0-0-0-0.compute-99.amazonaws.com'
  export CAMPFINBOT_MONGO_URL='127.0.0.1:12345'
  cd /home/ubuntu/nyt-campfinbot && /home/ubuntu/.virtualenvs/nyt-campfinbot/bin/python /home/ubuntu/nyt-campfinbot/campfinbot/bot.py
end script

nyt-campfinbot's People

Contributors

Stargazers

Watchers

Forkers

vocativ-dataviz said-and-done

nyt-campfinbot's Issues

logs -> papertrail

Should be really obvious when it's an amended filing.

Right now, just looks like a normal filing.

allow users to request a specific stat or candidate

campfinbot cand_name pulls up summary stats
campfinbot cand_name stat_name pulls up that stat

kill F24 notifier

it is just not that helpful and clutters things up

move whitelist/blacklist to campfin

we're currently blacklisting committees we don't want to alert inside campfinbot. Instead we should:

add an override field in campfin that is the final arbiter of whether to alert
otherwise alert all presidential candidates
...and all PACs with presidential IEs
set a few committees to override in campfin (DNC/RNC to yes, all minor candidates to no)

Food for thought: do we want new API endpoint(s) that only feed campfinbot or are there other campfin features that will take advantage of this distinction?

if a new committee gets added, EVERYTHING will be alerted

maybe we should check if the filing date as well before alerting? Or we could just be careful about the default sort of the API.

alert candidate with pac

say which candidate the pac is supporting

recovery if the api goes off line?

currently it seems that the bot crashes if it gets bad API data or the API is down. probably it should not.

Needs a logo.

notify Form99's

because they're sometimes interesting but often funny.

alert link to IE page instead of all IEs

alerting all F24s is filling up the channel. Alert a link to the IE page with a count of new IEs and maybe a list of all committees that have filed them instead.

new committees added to campfin

If a new committee is added or data is reloaded, the bot alerts them all.

Not totally sure the right way to fix this - maybe limiting "recent filings" to a few days from filing date?

exclude committees we don't care about

make it possible to exclude committees from the alert

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.