Code Monkey home page Code Monkey logo

fluentd-gelf-docker's Introduction

fluentd to graylog2 (gelf) forwarder

This will setup a multi container environment containing graylog2 and a fluentd GELF forwarder. It uses the official graylog2 and fluentd containers as base images. The distributed fluentd gelf plugin (plugins/out_gelf.rb) was developed by @emsearcy: https://github.com/emsearcy/fluent-plugin-gelf. It must be present to build the fluentd image successfully.

Using with docker-compose (recommended)

First, this will pull and execute graylog2 all-in-one container env. Then it will build and run a fluentd UPD forwarder to the graylog2 container

docker-compose up -d

Data persistence

By default running graylog with docker-compose up -d will persist data relatively in ./graylog2/data as configured in the docker-compose.yml file. You can simply modify that by placing a docker-compose.override.yml file with your specific configuration in the same directory.

graylog2:
  environment:
    GRAYLOG_NODE_ID: dev.local
  volumes:
    - {yourAbsoluteOrRelativPath}:/var/opt/greylog/data
    - {yourAbsoluteOrRelativPath}:/var/log/graylog

Make sure that you use docker-compose version 1.5.0 for overriding the docker-compose.yml configuration.

Manual setup steps

Setup graylog2 container

Run graylog2 all-in-one container env.

docker pull graylog2/allinone

docker run -t -p 9000:9000 -p 12201:12201 --name graylog2 -e GRAYLOG_NODE_ID=graylog2 graylog2/allinone

Now, you should be able to login to the graylog2 web interface http://host:9000/ (admin:admin)

Data persistence

To persist graylog2 data outside of your container you need to mount the data volumne.

docker run -t -p 9000:9000 -p 12201:12201 --name graylog2 -e GRAYLOG_NODE_ID=graylog2 -v {yourAbsolutePath}:/var/opt/graylog/data -v {yourAbsolutePath}:/var/log/graylog graylog2/allinone

Setup fluentd to graylog container

Build and link it to graylog2

docker build -t fluent-gelf .

docker run -p 24224:24224 --link graylog2:graylog2 -it --name fluent-gelf fluent-gelf

How to use

Now you're able to push any fluentd logs to HOST:24224. The fluentd messages will be forwarded to graylog2 as long as a tag in format gelf.app.XYZ is specified in the fluentd messages. Make sure, that you define a GELF UPD input source listening to port: 12201 (default) withing graylog2: http://host:9000/system/inputs

fluentd-gelf-docker's People

Contributors

onmomo avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

fluentd-gelf-docker's Issues

docs: how to persist data when using docker-compose.yml

I noticed you used proper instruction for running the containers separately, but the instructions for using docker-compose up -d assume additional parameters not in the docker-compose.yml file from this repo and not in your recommended instructions for using docker-compose.yml.

It looks like the example docker run command for the graylog2 uses information from Graylog2's Docker instructions. So, for consistency, to get the docker-compose up method to work the same way, I added a docker-compose.override.yml in my deployment with the following contents:

---
graylog2:
  environment:
    GRAYLOG_NODE_ID: dev-laptop-graylog2-0
    GRAYLOG_SERVER_SECRET: Gr4yL4p7Zy70x6P&^97494459230
  volumes:
    - ./graylog2/data:/var/opt/graylog/data
    - ./graylog2/log:/var/log/graylog

I just wanted to suggest at least mentioning the data persistence issue in the context of the docker-compose instructions, so inexperienced users of the Graylog2 image don't mistakenly believe your project has a broken Graylog2 image (when in fact the same issue exists with the Graylog2/allinone image from the Docker Hub).

I get System error: Unable to find user ubuntu

I down loaded the latest docker software for windows and cloned your repository, but when I ran docker-compose up -d I got the error message.

I was able to get the docker-compose up -d to run when I modified the Dockerfile, switching changing 'ubuntu' with 'root'.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.