The aim of this project is to develop end-to-end graph analytics module for big data. As a part of implementation, Stack Overflow Questions & Answers dataset, Neo4j Graph database, Spark's GraphX API, Scala programming and Amazon's EC2 cloud instance for hosting database for used.

The demonstration includes:

• Graph Cypher queries for the following use cases -
  • Finding trends of a technology in the data set
  • Identify top answerers for javascript questions
  • Fetch all the answers for each Java questions based on the scores
  • Lists where Else Were the Top Answerers of Java also Active?
  • Find users posting most Javascript questions
• Extended Graph Analytics using Scala based implementation for Spark's GraphX API for -
  • Evaluate an expert's rank for a programming language based on ranking using Page Rank Algorithm
  • Identifying the connected and non-connected tags from the dataset using Connected Components Algorithm.

• The experimental setup included:

  • Programming Language: Scala – Scala SDK – 4.7.0
  • Dependencies: Spark-core_2.11, Spark-sql_2.11, spark-graphx_2.11
  • Neo4j Database – Neo4j 3.2.4 version

• Cloud Implmentation for Neo4j Database: Neo4j Graph Database Community Edition was deployed on AWS EC2 instance and graph implementation for Stack overflow dataset. The Neo4j database edition deployed is limited to the single machine.

• Locally Neo4j Community Edition can be downloaded from http://neo4j.com/download/ and server should be started after installation. Default graphdb folder should be replaced with unzipped folder. Consequently, restart the services.

• Clone the SparkNeo4j locally, build the Gradle and Run the project to execute

GraphX