eyeseast / propublica-congress Goto Github PK

View Code? Open in Web Editor NEW

54.0 5.0 19.0 74 KB

A Python client for the ProPublica Congress API

Home Page: https://propublica-congress.readthedocs.io/en/latest/

License: MIT License

Python 100.00%

politics congress government api api-client python

propublica-congress's Introduction

Python Congress

A Python client for the ProPublica Congress API

Install

From PyPI:

pip install python-congress

Download and run the install script:

git clone https://github.com/eyeseast/propublica-congress && cd propublica-congress
python setup.py install

Usage

>>> from congress import Congress
>>> congress = Congress(API_KEY)

# get member by bioguide ID
>>> pelosi = congress.members.get('P000197')
>>> pelosi['twitter_id']
'NancyPelosi'

# get recent House bills
# recent bills come in two types: 'introduced' and 'updated'
>>> introd = congress.bills.recent(chamber='house', congress=111, type='introduced')
>>> len(introd['bills'])
20
>>> introd['chamber']
'House'

# or use a convenience function
>>> introd = congress.bills.introduced('house')
>>> introd['chamber']
'House'
>>> len(introd['bills'])
20

propublica-congress's People

Stargazers

Watchers

Forkers

aaghevli upliftagency hackshackersdc giltolley dadiletta htrajan astrowonk webcandyllc rumtumtum ejafek13 boulas627 tayler-erbe wethepeopleonline meli-lewis reed-schimmel robertbarrett shhawkins

propublica-congress's Issues

MembersClient.filter should return a list of members

Currently returning a single dictionary. The parse param strikes again.

Had issue with http2

FYI I'm using python2 and I had an issue querying propublica using your http2 query.

I get an error of:

SSLHandshakeError: [SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:590)

I fixed the error by switching the querying method to urllib2. If other people are having the same issue as me happy to commit the change, otherwise if you've got a reason for using http2 that I'm not aware of we can keep it as is.

Update tests

Tests still point to the old NYT API, so fix that.

Go black

Save me from myself: https://github.com/ambv/black

Proper documentation

Either need an extensive readme, with doctests, or Sphinx.

Support pagination in bills endpoint and cleanup bills functionality

Current bills endpoint seems to only get the 20 latest introduced / updated bills. Adding pagination (via ProPublica's offset parameter) will allow the API caller to get earlier bills.

Also, the API no longer supports 'major' bills. The type of bill to query for can be among: introduced, updated, active, passed, enacted or vetoed.

Error handling

Handle argument errors early, before making queries (like chamber args)
Catch errors before running any parse function in Client.fetch.

Get members by state/district

This has new endpoints: https://propublica.github.io/congress-api-docs/#get-current-members-by-state-district

Document how to add new endpoints

There's a bunch of new stuff in the API. If it's clear how to add support, hopefully more people will send PRs.

Socket warnings in Python 3 tests

test_bill_detail (test.BillTest) ... /home/travis/virtualenv/python3.6.1/lib/python3.6/site-packages/httplib2/__init__.py:858: DeprecationWarning: key_file, cert_file and check_hostname are deprecated, use a custom context instead.

  check_hostname=disable_ssl_certificate_validation ^ True)

ok

/opt/python/3.6.1/lib/python3.6/unittest/suite.py:107: ResourceWarning: unclosed <ssl.SSLSocket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('172.17.0.8', 53458), raddr=('52.84.145.112', 443)>

  for index, test in enumerate(self):

/opt/python/3.6.1/lib/python3.6/unittest/suite.py:107: ResourceWarning: unclosed <ssl.SSLSocket fd=6, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('172.17.0.8', 37272), raddr=('52.84.145.100', 443)>

  for index, test in enumerate(self):

test_bill_subjects (test.BillTest) ... ok

/opt/python/3.6.1/lib/python3.6/unittest/suite.py:107: ResourceWarning: unclosed <ssl.SSLSocket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('172.17.0.8', 37330), raddr=('52.84.145.96', 443)>

  for index, test in enumerate(self):

/opt/python/3.6.1/lib/python3.6/unittest/suite.py:107: ResourceWarning: unclosed <ssl.SSLSocket fd=6, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('172.17.0.8', 60592), raddr=('52.84.145.83', 443)>

  for index, test in enumerate(self):

test_bills_by_member (test.BillTest) ... ok

/opt/python/3.6.1/lib/python3.6/unittest/suite.py:107: ResourceWarning: unclosed <ssl.SSLSocket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('172.17.0.8', 54062), raddr=('52.84.145.4', 443)>

  for index, test in enumerate(self):

/opt/python/3.6.1/lib/python3.6/unittest/suite.py:107: ResourceWarning: unclosed <ssl.SSLSocket fd=6, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('172.17.0.8', 51628), raddr=('52.84.145.218', 443)>

  for index, test in enumerate(self):

test_recent_bills (test.BillTest) ... ok

/opt/python/3.6.1/lib/python3.6/unittest/suite.py:84: ResourceWarning: unclosed <ssl.SSLSocket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('172.17.0.8', 37828), raddr=('52.84.145.211', 443)>

  return self.run(*args, **kwds)

/opt/python/3.6.1/lib/python3.6/unittest/suite.py:84: ResourceWarning: unclosed <ssl.SSLSocket fd=6, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('172.17.0.8', 37186), raddr=('52.84.145.135', 443)>

  return self.run(*args, **kwds)

test_generic_fetch (test.ClientTest) ... ok

/opt/python/3.6.1/lib/python3.6/unittest/suite.py:84: ResourceWarning: unclosed <ssl.SSLSocket fd=6, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('172.17.0.8', 37306), raddr=('52.84.145.100', 443)>

  return self.run(*args, **kwds)

/opt/python/3.6.1/lib/python3.6/unittest/suite.py:84: ResourceWarning: unclosed <ssl.SSLSocket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('172.17.0.8', 53492), raddr=('52.84.145.112', 443)>

  return self.run(*args, **kwds)

test_committee_detail (test.CommitteeTest) ... ok

/opt/python/3.6.1/lib/python3.6/unittest/suite.py:107: ResourceWarning: unclosed <ssl.SSLSocket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('172.17.0.8', 37362), raddr=('52.84.145.96', 443)>

  for index, test in enumerate(self):

/opt/python/3.6.1/lib/python3.6/unittest/suite.py:107: ResourceWarning: unclosed <ssl.SSLSocket fd=6, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('172.17.0.8', 60624), raddr=('52.84.145.83', 443)>

  for index, test in enumerate(self):

test_committee_list (test.CommitteeTest) ... ok

/opt/python/3.6.1/lib/python3.6/unittest/suite.py:84: ResourceWarning: unclosed <ssl.SSLSocket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('172.17.0.8', 54092), raddr=('52.84.145.4', 443)>

  return self.run(*args, **kwds)

/opt/python/3.6.1/lib/python3.6/unittest/suite.py:84: ResourceWarning: unclosed <ssl.SSLSocket fd=6, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('172.17.0.8', 51646), raddr=('52.84.145.218', 443)>

  return self.run(*args, **kwds)

test_django_cache (test.DjangoTest) ... ok

test_404 (test.ErrorTest) ... ok

/opt/python/3.6.1/lib/python3.6/unittest/suite.py:107: ResourceWarning: unclosed <ssl.SSLSocket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('172.17.0.8', 37856), raddr=('52.84.145.211', 443)>

  for index, test in enumerate(self):

test_bad_vote_args (test.ErrorTest) ... ok

test_no_chamber_args (test.ErrorTest) ... ok

test_compare_members (test.MemberTest) ... ok

/opt/python/3.6.1/lib/python3.6/unittest/suite.py:107: ResourceWarning: unclosed <ssl.SSLSocket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('172.17.0.8', 37214), raddr=('52.84.145.135', 443)>

  for index, test in enumerate(self):

/opt/python/3.6.1/lib/python3.6/unittest/suite.py:107: ResourceWarning: unclosed <ssl.SSLSocket fd=6, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('172.17.0.8', 53522), raddr=('52.84.145.112', 443)>

  for index, test in enumerate(self):

test_departing_members (test.MemberTest) ... ok

/opt/python/3.6.1/lib/python3.6/unittest/suite.py:107: ResourceWarning: unclosed <ssl.SSLSocket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('172.17.0.8', 37338), raddr=('52.84.145.100', 443)>

  for index, test in enumerate(self):

/opt/python/3.6.1/lib/python3.6/unittest/suite.py:107: ResourceWarning: unclosed <ssl.SSLSocket fd=6, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('172.17.0.8', 37396), raddr=('52.84.145.96', 443)>

  for index, test in enumerate(self):

test_filter_members (test.MemberTest) ... ok

/opt/python/3.6.1/lib/python3.6/unittest/suite.py:107: ResourceWarning: unclosed <ssl.SSLSocket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('172.17.0.8', 60658), raddr=('52.84.145.83', 443)>

  for index, test in enumerate(self):

/opt/python/3.6.1/lib/python3.6/unittest/suite.py:107: ResourceWarning: unclosed <ssl.SSLSocket fd=6, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('172.17.0.8', 54128), raddr=('52.84.145.4', 443)>

  for index, test in enumerate(self):

test_get_member (test.MemberTest) ... ok

/opt/python/3.6.1/lib/python3.6/unittest/suite.py:107: ResourceWarning: unclosed <ssl.SSLSocket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('172.17.0.8', 51684), raddr=('52.84.145.218', 443)>

  for index, test in enumerate(self):

/opt/python/3.6.1/lib/python3.6/unittest/suite.py:107: ResourceWarning: unclosed <ssl.SSLSocket fd=6, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('172.17.0.8', 37882), raddr=('52.84.145.211', 443)>

  for index, test in enumerate(self):

test_new_members (test.MemberTest) ... ok

/opt/python/3.6.1/lib/python3.6/unittest/suite.py:84: ResourceWarning: unclosed <ssl.SSLSocket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('172.17.0.8', 37242), raddr=('52.84.145.135', 443)>

  return self.run(*args, **kwds)

/opt/python/3.6.1/lib/python3.6/unittest/suite.py:84: ResourceWarning: unclosed <ssl.SSLSocket fd=6, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('172.17.0.8', 53550), raddr=('52.84.145.112', 443)>

  return self.run(*args, **kwds)

test_nomination_detail (test.NominationTest) ... ok

/opt/python/3.6.1/lib/python3.6/unittest/suite.py:107: ResourceWarning: unclosed <ssl.SSLSocket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('172.17.0.8', 37370), raddr=('52.84.145.100', 443)>

  for index, test in enumerate(self):

/opt/python/3.6.1/lib/python3.6/unittest/suite.py:107: ResourceWarning: unclosed <ssl.SSLSocket fd=6, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('172.17.0.8', 37428), raddr=('52.84.145.96', 443)>

  for index, test in enumerate(self):

test_nomination_list (test.NominationTest) ... ok

/opt/python/3.6.1/lib/python3.6/unittest/suite.py:107: ResourceWarning: unclosed <ssl.SSLSocket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('172.17.0.8', 60690), raddr=('52.84.145.83', 443)>

  for index, test in enumerate(self):

/opt/python/3.6.1/lib/python3.6/unittest/suite.py:107: ResourceWarning: unclosed <ssl.SSLSocket fd=6, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('172.17.0.8', 54158), raddr=('52.84.145.4', 443)>

  for index, test in enumerate(self):

test_nominations (test.NominationTest) ... ok

/opt/python/3.6.1/lib/python3.6/unittest/suite.py:107: ResourceWarning: unclosed <ssl.SSLSocket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('172.17.0.8', 51738), raddr=('52.84.145.218', 443)>

  for index, test in enumerate(self):

/opt/python/3.6.1/lib/python3.6/unittest/suite.py:107: ResourceWarning: unclosed <ssl.SSLSocket fd=6, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('172.17.0.8', 37942), raddr=('52.84.145.211', 443)>

  for index, test in enumerate(self):

test_nominations_by_state (test.NominationTest) ... ok

/opt/python/3.6.1/lib/python3.6/unittest/suite.py:84: ResourceWarning: unclosed <ssl.SSLSocket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('172.17.0.8', 37300), raddr=('52.84.145.135', 443)>

  return self.run(*args, **kwds)

/opt/python/3.6.1/lib/python3.6/unittest/suite.py:84: ResourceWarning: unclosed <ssl.SSLSocket fd=6, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('172.17.0.8', 53608), raddr=('52.84.145.112', 443)>

  return self.run(*args, **kwds)

test_congress_years (test.UtilTest) ... ok

test_vote_rollcall (test.VoteTest) ... ok

/opt/python/3.6.1/lib/python3.6/unittest/suite.py:107: ResourceWarning: unclosed <ssl.SSLSocket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('172.17.0.8', 37424), raddr=('52.84.145.100', 443)>

  for index, test in enumerate(self):

/opt/python/3.6.1/lib/python3.6/unittest/suite.py:107: ResourceWarning: unclosed <ssl.SSLSocket fd=6, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('172.17.0.8', 37490), raddr=('52.84.145.96', 443)>

  for index, test in enumerate(self):

test_votes_by_date (test.VoteTest) ... ok

/opt/python/3.6.1/lib/python3.6/unittest/suite.py:107: ResourceWarning: unclosed <ssl.SSLSocket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('172.17.0.8', 60760), raddr=('52.84.145.83', 443)>

  for index, test in enumerate(self):

/opt/python/3.6.1/lib/python3.6/unittest/suite.py:107: ResourceWarning: unclosed <ssl.SSLSocket fd=6, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('172.17.0.8', 54228), raddr=('52.84.145.4', 443)>

  for index, test in enumerate(self):

test_votes_by_date_range (test.VoteTest) ... ok

/opt/python/3.6.1/lib/python3.6/unittest/suite.py:107: ResourceWarning: unclosed <ssl.SSLSocket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('172.17.0.8', 51782), raddr=('52.84.145.218', 443)>

  for index, test in enumerate(self):

/opt/python/3.6.1/lib/python3.6/unittest/suite.py:107: ResourceWarning: unclosed <ssl.SSLSocket fd=6, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('172.17.0.8', 37998), raddr=('52.84.145.211', 443)>

  for index, test in enumerate(self):

test_votes_by_month (test.VoteTest) ... ok

/opt/python/3.6.1/lib/python3.6/unittest/suite.py:107: ResourceWarning: unclosed <ssl.SSLSocket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('172.17.0.8', 37372), raddr=('52.84.145.135', 443)>

  for index, test in enumerate(self):

/opt/python/3.6.1/lib/python3.6/unittest/suite.py:107: ResourceWarning: unclosed <ssl.SSLSocket fd=6, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('172.17.0.8', 53692), raddr=('52.84.145.112', 443)>

  for index, test in enumerate(self):

test_votes_by_reversed_range (test.VoteTest) ... ok

/opt/python/3.6.1/lib/python3.6/unittest/suite.py:107: ResourceWarning: unclosed <ssl.SSLSocket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('172.17.0.8', 37518), raddr=('52.84.145.100', 443)>

  for index, test in enumerate(self):

test_votes_by_type (test.VoteTest) ... ok

/opt/python/3.6.1/lib/python3.6/unittest/suite.py:107: ResourceWarning: unclosed <ssl.SSLSocket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('172.17.0.8', 37606), raddr=('52.84.145.96', 443)>

  for index, test in enumerate(self):

/opt/python/3.6.1/lib/python3.6/unittest/suite.py:107: ResourceWarning: unclosed <ssl.SSLSocket fd=6, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('172.17.0.8', 60894), raddr=('52.84.145.83', 443)>

  for index, test in enumerate(self):

test_votes_today (test.VoteTest) ... ok

/opt/python/3.6.1/lib/python3.6/unittest/suite.py:84: ResourceWarning: unclosed <ssl.SSLSocket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('172.17.0.8', 54392), raddr=('52.84.145.4', 443)>

  return self.run(*args, **kwds)

/opt/python/3.6.1/lib/python3.6/unittest/suite.py:84: ResourceWarning: unclosed <ssl.SSLSocket fd=6, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('172.17.0.8', 51956), raddr=('52.84.145.218', 443)>

  return self.run(*args, **kwds)



----------------------------------------------------------------------

Ran 28 tests in 23.495s

OK

travis_time:end:10656ca0:start=1495655127550382035,finish=1495655151528116692,duration=23977734657
�[0K

�[32;1mThe command "python setup.py test" exited with 0.�[0m

Done. Your build exited with 0.

Use URI templates

This would probably speed up adding new endpoints: https://pypi.org/project/uritemplate/

Return objects instead of JSON

Because @hobbes7878 asked.

Also something I've been thinking about, especially if it helps interlink parts of the API. Get a member, then get that member's votes, bills, etc. Worth thinking about.

LICENSE

Should be MIT probably.

Travis CI

Cut a new release

Tried doing some prototyping with this earlier today, and congress.votes.VotesClient.recent wasn't available when I used the version on PyPI.

Turns out what's published there (v0.3.5) is missing all of this subsequent development: v0.3.5...e13a62b

Perhaps worth a v0.3.6?

Break into modules

Getting to be one big file doing lots of things.

Consolidate 'parse' functions and functionality

Client.fetch takes a parse parameter, which should be a function that takes the raw JSON response from the API and returns something slightly cleaner.

Right now, it's a lambda that gets passed around or overridden on a method-by-method bases.

There are only two parse functions now, and they're almost identical:

lambda r: r['results'][0]
lambda r: r['results']

This is annoying to keep track of, and there's no reason we can't use a smarter function. It's also potentially an impediment to #15.

I think it makes more sense to have a parse method on Client, which a subclass can override. This would be a good hook for instantiating a class.

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.