Latent Dirichlet Allocation Based Recommender System

This is a prototype of a "Semantics-aware" recommender system built on top of LDA, Latent Dirichelet Allocation. A high-level workflow is described in this presentation. This recommender system gives very nice results within a limited training time.

Preparation work

First of all you need to web-crawl any news website or have a corpus of documents. Each document should be like the format below:

Line	Sample content	Require?
1	http://www.bbc.com/news/entertainment-arts-31164553	Optional
2	Baz Luhrmann to make Netflix musical	Required
3	Moulin Rouge director Baz Luhrmann is to make a 13-part musical series for streaming service Netflix.Set in 1970s New York, The Get Down traces how the city gave birth to disco, new wave and hip-hop.It will follow a group of teenagers from downtrodden, crime-ridden South Bronx who find solace in each other and their own musical abilities.Luhrmann will direct the first two episodes, echoing David Fincher's input to Netflix's hit series House Of Cards.	Required

A basic corpus of web-crawled BBC News is provided in the repo directory.

Usage

Make sure you have gensim & nltk installed in your python path, this repo depeneds on these libraries.

Build model

python /path-to-repo/LDAModel_English.py --dir /path-to-repo/BBCNews

Model building should take minutes to run, depends on how large your corpus is.

After the model is generated and saved, now try to get some recommendations:

python /path-to-repo/LDAModel_English.py --shell True

You should see something like this image. Enter any post id (positive integer) and you'll be recommended articles based on that id. You may also type comma-separated ids e.g. 123,234,345 to get recommendations based on that combination.