Code Monkey home page Code Monkey logo

emergencypredictiononspark's Introduction

OpenPower- Emergency Prediction using Spark

An estimated 240 million calls to 911 are made each year, in the U.S alone. Firefighters constitute a significant number of first responders to these emergency calls. Predicting frequency of emergency events is extremely useful, as it allows better fire deparment logistics. This incurs in not only a reductions of costs, but it also helps saving more lives.

This project demontrates how deep learning can be used to predict amount of emergency calls, given different types of emergency events and multiple fire stations. Our method relies on Apache Spark for dataprocessing, making it possible to scale across multiple machines. We also provide optional GPU processing for some operations.

Data used for our experiments is public, and are made available through Seattle Open Data portal. The datasets used in our experiments are the following:

Requirements

This package is compatible with Spark 1.5+ and scala 2.10

Spark Version Scala Version Compatible version of Spark GPU
1.5+ 2.10 1.0.0

GPU requirements are the following:

CUDA Toolkit Version
7.5

Setup

Before running this project, set following environment variables accordingly:

Variable Value Example
SPARK_HOME <path to Apache Spark installation> /home/user/apache-spark
MASTER <URL to SPARK Master> spark://192.168.0.1:7077
HADOOP_HOME <path to Apache Hadoop installation> /home/user/hadoop

See also example_set_environment.sh.

Building

(Note: Not necessary if using pre-compiled release version)

Requires SBT. Also make sure nvcc is visible through the PATH environment variable.

Just excute the following commands:

  • cd fireplanning
  • ./build.sh

Uploading Seattle emergency data to Hadoop

This step is necessary before running this application. Make sure HADOOP_HOME has been correctly set before uploading data

  • cd fireplanning
  • ./submit_data.sh

Running

After uploading data into Hadoop, run with the following commands:

  • cd fireplanning
  • ./run.sh

This shell script will run our main application through Spark Shell in interactive mode. This means that after execution, the application will not exit, and scala terminal will be available to user.

Execution logs

Sample execution logs are available under execution-log-samples folder.

emergencypredictiononspark's People

Contributors

marcojrfurtado avatar

Watchers

James Cloos avatar Jonathan Andrade avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.