kafka-streams

This is the repository for the examples of using Kafka streams covered in the blog posts:

Requirements to build this project

Java 8
Gradle

Requirements to run the examples

kafka version kafka_2.11-0.10.1.0 see the section marked "Running a task on a particular version of Scala"
The json-data-generator from ACES,Inc

Setup Instructions

Extact the kafka_2.11-0.10.1.0.tgz file

tar -xvzf kafka_2.11-0.10.1.0.tgz

Start zookeeper and kafka

      kafka-install-dir/bin/zookeeper-server-start.sh kafka-install-dir/conf/zookeeper.properties
      kafka-install-dir/bin/kafka-server-start.sh kafka-install-dir/conf/server.properties

Install the Json-Data-Generator

Download the latest json-data-generator release and follow the install instructions here

Setup the kafka-streams repo

Clone or fork the repo

     git clone [email protected]:bbejeck/kafka-streams    
     cd kafka-streams

Then copy the json config files to json generator conf directory

    cp streaming-workflows/* <dir>/json-data-generator-1.2.0/conf

Create all the topics required by the examples

     ./bin/create-topics.sh /usr/local/kafka_2.11-0.10.1.0 localhost 2181
     args are kafka home, zookeeper host and zookeeper port adjust accordingly

Running the Purchase Processor API KStreams API Examples

 cd <dir>/json-data-generator-1.2.0/
 java -jar json-data-generator-1.2.0 purchases-config.json
 cd kafka-streams
 ./gradlew runPurchaseProcessor | runPurchaseStreams

Running the Stock Trades Processor API or KStreams API Examples

 cd <dir>/json-data-generator-1.2.0/
 java -jar json-data-generator-1.2.0 stock-transactions-config.json
 cd kafka-streams
 ./gradlew runStockProcessor | runStockStreams

Running the Twitter KStreams Language Classification Example

rename src/main/resources/twitter-app.properties.template to twitter-app.properties 
fill out the properties file with all the required values

cd kafka-streams
./gradlew runTwitterKstreamNLP

Viewing the results of the purchase streaming examples

cd kafka_install-dir/bin
./kafka-console-consumer --topic [patterns|rewards|purchases] --zookeeper localhost:2181

Viewing the results of the stock-trading streaming examples

cd kafka_install-dir/bin
./kafka-console-consumer --topic [stocks-out|transaction-summary] --zookeeper localhost:2181

Viewing the results of the Twitter KStreams Language Classification Example

cd kafka_install-dir/bin
./kafka-console-consumer --topic [english|french|spanish] --zookeeper localhost:2181

bbejeck / kafka-streams Goto Github PK

kafka-streams's Introduction

kafka-streams

Requirements to build this project

Requirements to run the examples

Setup Instructions

Extact the kafka_2.11-0.10.1.0.tgz file

Start zookeeper and kafka

Install the Json-Data-Generator

Setup the kafka-streams repo

Running the Purchase Processor API KStreams API Examples

Running the Stock Trades Processor API or KStreams API Examples

Running the Twitter KStreams Language Classification Example

Viewing the results of the purchase streaming examples

Viewing the results of the stock-trading streaming examples

Viewing the results of the Twitter KStreams Language Classification Example

kafka-streams's People

Contributors

Stargazers

Watchers

Forkers

kafka-streams's Issues

Recommend Projects

Recommend Topics

Recommend Org