timini / hansard Goto Github PK

• Do individual people cluster into their political parties?
  • get party membership data for all speakers
  • create vector representations of speeches
  • cluster speeches using PCA and label by party http://scikit-learn.org/stable/modules/decomposition.html#decompositions
• Do speeches / people from the house of lords vs the house of commons cluster separately?
  • PCA of all speeches, labeled by house.
• What are the significant terms for the house of lords vs commons?
• What topics do one house prefer to talk about over the other?
  • generate the topics for house of lords
  • generate the topics for the house of commons
• What topics do each party prefer to talk about?
• Scrape news articles and try to classify using our trained classifiers

The Site is based on Topics.

User follows topics.

They are recommended politicians who talk about these topics.

When a new comment is made which falls into that topic they are notified.

http://www.data.parliament.uk/dataset/members-of-the-house-of-commons

http://explore.data.parliament.uk/?learnmore=Members
http://lda.data.parliament.uk/members.json
http://lda.data.parliament.uk/members/172.json

http://www.data.parliament.uk/dataset/constituencies

http://explore.data.parliament.uk/?learnmore=Constituencies

http://lda.data.parliament.uk/constituencies/143462.json
http://lda.data.parliament.uk/constituencies.json

https://github.com/cemoody/lda2vec

Towards Sentiment Analysis on Parliamentary Debates in Hansard

Parli-N-Grams (slides)

andrew whitby's project

pyldavis is not working with python 2, plus gensim prefers python 3.

Time to update

To get the project started we should start with a simple API for the basic entities such as

• MPs
• Constituencies
• #13 Add the relationship between the MPs and Constituencies
• Debates API
• #14 Topics API
• Information Extraction for Debates API
  • convert comments_small.json to CSV to import into iepy

• words whose frequency have the largest differences in the house of commons vs the lords
• words whose frequency have the largest differences between the political parties.

• go with docker-compose on AWS
• Is it possible to run tests on travis across the stack?