information-retrieval's Introduction

Information-Retrieval

Topic Modelling and Clustering

https://github.com/snmsoodan/Information-Retrieval/tree/master/IR/hw8

Performed LDA on a dataset of 85k documents to obtain representation of all documents into 200 topics using Machine Learning libraries in Java-Mallet.Then applied Density-Based Clustering using Python scikit-learn library, where every document gets assigned exactly one cluster out of total 25 clusters.Compared and achieved partitioning with existing query-document assignment for 25 queries to create a confusion matrix,achieving 88% accuracy.

Spam Classifier

https://github.com/snmsoodan/Information-Retrieval/tree/master/IR/hw5

Developed a spam classifier using logistic regression.The classifier was trained on 65000 documents and achieved accuracy of 99% for prediction of 15000 documents.

Page Rank

https://github.com/snmsoodan/Information-Retrieval/tree/master/IR/hw4

Implemented PageRank, a link analysis algorithm, in Java to get top 100 Webpages from the crawled data. Implemented HITS, Hyperlink-Induces Topic Search, which estimates the authority and the hub value of each crawled page.

Web Crawler

https://github.com/snmsoodan/Information-Retrieval/tree/master/IR/hw3

The Web Crawler, systematically crawled the world wide web to construct a relevant document collection focused on a particular topic and then created a vertical search engine using ElasticSearch. Crawled from seed page up to specified maximum depth by implementing politeness policy

Recommend Projects

snmsoodan / information-retrieval Goto Github PK