Code Monkey home page Code Monkey logo

flume-stackexchange-source's Introduction

Flume StackExchange Source

The StackExchange source queries StackExchange API and passes information to the flume agent to which it belongs.

Configuration

This is an example of flume-conf.properties used for testing this source.

# Configuration for Flume agent a1
a1.sources = s1
a1.channels = c1
a1.sinks = k1

# This is standard Source config
a1.sources.s1.type = it.miriade.flume.source.stackexchange.StackExchangeSource
a1.sources.s1.name = Miriade-FlumeStackExchangeSource

# This is StackExchange Source config
a1.sources.s1.stackexchange.api.poller.fixedrate = 2500
a1.sources.s1.stackexchange.api.poller.timeunit = MILLISECONDS

# Specify the channel the source should use
a1.sources.s1.channels = c1

# This channel works in memory, it's use for testing
a1.channels.c1.type = memory
a1.channels.c1.capacity = 100000

# This sink writes on a file in /tmp directory, it's use for testing
a1.sinks.k1.type = file_roll
a1.sinks.k1.sink.directory = /tmp
a1.sinks.k1.rollInterval = 60

# Specify the channel the sink should use
a1.sinks.k1.channel = c1

About Flume

For further information see http://flume.apache.org/FlumeUserGuide.html.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.