Synopsis
This is an Hadoop Map/Reduce application for Working on weather data It reads the text input files, breaks each line into stations weather data and finds average for temperature , dew point , wind speed. The output is a locally sorted list of stations and its 12 attribute vector of average temperature , dew , wind speed of 4 sections for each month.
Installation
Since this a Map reduce Project , please install Apache Hadoop , Refere the below site for more details
Once you have installed the hadoop , download the project using git commands and install it.
$ git clone https://github.com/vasanth-mahendran/weather-data-hadoop.git $ cd weather-data-hadoop $ mvn install
Run
Add the sample input file given in the proect to the hdfs
hdfs dfs -put sample_weather.txt
and start the Map reduce application to submit the job to hadoop
hadoop jar ./target/weather-1.0.jar sample_weather.txt dataOutput