Code Monkey home page Code Monkey logo

docker-hadoop's Introduction

docker-hadoop

A standalone single-node Hadoop image for Docker

  1. Build the Docker image and run it

    1. Inside the root directory, run this command to create the image:

    docker build . -t hadoop:1

    1. Inside the root directory, run this command to initiate the container:

    docker run -it --name hadoop --rm -p 50070:50070 -p 8088:8088 hadoop:1

    1. Check web interfaces of Hadoop:

    http://localhost:50070/explorer.html#/example

    http://localhost:8088/cluster

  2. Compile the MapReduce program and run it inside Hadoop prompt

    1. You can run this to start the PI Calculation example:

    $ /root/hadoop/bin/hadoop jar /root/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-examples-3.1.1.jar pi 1 10

    1. Copy sample file into HDFS input directory:

    $ /root/hadoop/bin/hadoop fs -mkdir -p /example

    $ /root/hadoop/bin/hadoop fs -put /root/example/criptomoedas.txt /example

    1. Setup Java environment variables:

    $ export JAVA_HOME=/usr/java/latest

    $ export PATH=${JAVA_HOME}/bin:${PATH}

    $ export HADOOP_CLASSPATH=${JAVA_HOME}/lib/tools.jar

    1. Compile MapReduce Java job into a .jar file:

    $ /root/hadoop/bin/hadoop com.sun.tools.javac.Main /root/example/WordCount.java

    $ cd root/example

    $ jar cf wc.jar WordCount*.class

    $ cd ..

    $ cd ..

    $ jar tvf /root/example/wc.jar

    1. Run the job:

    $ /root/hadoop/bin/hadoop jar /root/example/wc.jar WordCount /example/criptomoedas.txt /example/criptomoedas_mapreduce_result

    1. Show the results:

    $ /root/hadoop/bin/hadoop fs -cat /example/criptomoedas_mapreduce_result/part-00000

docker-hadoop's People

Contributors

marcelohama avatar

Stargazers

 avatar

Watchers

 avatar  avatar  avatar  avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.