Code Monkey home page Code Monkey logo

tinysyslog's Introduction

tinysyslog Go Report Card codecov

A tiny and simple syslog server with log rotation. tinysyslog was born out of the need for a tiny, easy to set up and use syslog server that simply writes every incoming log (in RFC 5424 format only) to a file that is automatically rotated, to stdout or stderr (mostly for Docker) and or to Elasticsearch. tinysyslog is based on go-syslog and lumberjack.

Quickstart

git clone https://github.com/alexferl/tinysyslog.git
cd tinysyslog
make run

If tinysyslog started properly you should see:

2023-08-30T18:38:09-04:00 INF server.go:52 > tinysyslog starting
2023-08-30T18:38:09-04:00 INF server.go:63 > tinysyslog listening on 127.0.0.1:5140

You can take make sure logs are processed by the server by entering the following in a terminal:

nc -w0 -u 127.0.0.1 5140 <<< '<165>1 2016-01-01T12:01:21Z hostname appname 1234 ID47 [exampleSDID@32473 iut="9" eventSource="test" eventID="123"] message'

You should then see the following output in your terminal:

Jan  1 12:01:21 hostname appname[1234]: message

Docker Quickstart

Download the image:

docker pull admiralobvious/tinysyslog

Start the container:

docker run --rm --name tinysyslog -p 5140:5140/udp -d admiralobvious/tinysyslog

Send a log:

nc -w0 -u 127.0.0.1 5140 <<< '<165>1 2016-01-01T12:01:21Z hostname appname 1234 ID47 [exampleSDID@32473 iut="9" eventSource="test" eventID="123"] message'

Confirm the container received it:

docker logs tinysyslog
2023-08-30T22:46:06Z INF build/server.go:52 > tinysyslog starting
2023-08-30T22:46:06Z INF build/server.go:63 > tinysyslog listening on 0.0.0.0:5140
Jan  1 12:01:21 hostname appname[1234]: message

Kubernetes Quickstart

Apply the manifest to your cluster:

kubectl apply -f kubernetes/tinysyslog.yaml

Make sure the container is running:

kubectl get pods | grep tinysyslog
tinysyslog-6c85886f65-q9cxw          1/1       Running   0          1m

Confirm the pod started properly:

kubectl logs tinysyslog-6c85886f65-q9cxw
2023-08-30T22:46:06Z INF build/server.go:52 > tinysyslog starting
2023-08-30T22:46:06Z INF build/server.go:63 > tinysyslog listening on 0.0.0.0:5140

You can now send logs from your app(s) to tinysyslog:5140.

Configuration

Usage of ./tinysyslogd:
      --app-name string                           The name of the application. (default "tinysyslog")
      --bind-addr string                          IP and port to listen on. (default "127.0.0.1:5140")
      --env-name string                           The environment of the application. Used to load the right configs file. (default "PROD")
      --filter string                             Filter to filter logs with. Valid filters: [regex]
      --filter-regex string                       Regex to filter with.
      --log-level string                          The granularity of log outputs. Valid levels: [PANIC FATAL ERROR WARN INFO DISABLED TRACE DISABLED] (default "INFO")
      --log-output string                         The output to write to. Valid outputs: [stdout stderr] (default "stdout")
      --log-writer string                         The log writer. Valid writers: [console json] (default "console")
      --mutator string                            Mutator type to use. Valid mutators: [text json] (default "text")
      --sink-console-output string                Console to output to. Valid outputs: [stdout stderr] (default "stdout")
      --sink-elasticsearch-addresses strings      Elasticsearch server addresses.
      --sink-elasticsearch-api-key string         Elasticsearch api key.
      --sink-elasticsearch-cloud-id string        Elasticsearch cloud id.
      --sink-elasticsearch-index-name string      Elasticsearch index name. (default "tinysyslog")
      --sink-elasticsearch-password string        Elasticsearch password.
      --sink-elasticsearch-service-token string   Elasticsearch service token.
      --sink-elasticsearch-username string        Elasticsearch username.
      --sink-filesystem-filename string           File path to write incoming logs to. (default "syslog.log")
      --sink-filesystem-max-age int               Maximum age (in days) before a log is deleted. (default 30)
      --sink-filesystem-max-backups int           Maximum backups to keep. (default 10)
      --sink-filesystem-max-size int              Maximum log size (in megabytes) before it's rotated. (default 100)
      --sinks strings                             Sinks to save syslogs to. Valid sinks: [console elasticsearch filesystem] (default [console])
      --socket-type string                        Type of socket to use, TCP or UDP. If no type is specified, both are used.

tinysyslog's People

Contributors

admiralobvious avatar alexferl avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

tinysyslog's Issues

Mikrotik logs display as blank

Hi,

Syslogs from Mikrotik routers show like this:

tinysyslog  | time="2024-01-21T20:37:53Z" level=info msg="tinysyslog listening on 0.0.0.0:5140"
tinysyslog  | Jan  1 00:00:00  []: 
tinysyslog  | Jan  1 00:00:00  []: 
tinysyslog  | Jan  1 00:00:00  []: 
tinysyslog  | Jan  1 00:00:00  []: 
tinysyslog  | Jan  1 00:00:00  []: 
tinysyslog  | Jan  1 00:00:00  []: 
tinysyslog  | Jan  1 00:00:00  []: 
tinysyslog  | Jan  1 00:00:00  []: 
tinysyslog  | Jan  1 00:00:00  []: 
tinysyslog  | Jan  1 00:00:00  []: 
tinysyslog  | Jan  1 00:00:00  []: 
tinysyslog  | Jan  1 00:00:00  []: 
tinysyslog  | Jan  1 00:00:00  []: 

A tcpdump of the UDP packet shows that it does contain the log message.

Regex filter not being loaded correctly?

The grok pattern seems to load:

# docker run --rm --name tinysyslog -p 2514:5140/udp admiralobvious/tinysyslog --log-level debug --filter-grok-pattern test --filter grok
time="2020-01-31T00:34:39Z" level=info msg="tinysyslog starting"
time="2020-01-31T00:34:39Z" level=debug msg="Using mutator type 'text'"
time="2020-01-31T00:34:39Z" level=debug msg="Using filter type 'grok' with pattern 'test'"
time="2020-01-31T00:34:39Z" level=debug msg="Adding sink type 'console'"
time="2020-01-31T00:34:39Z" level=info msg="tinysyslog listening on 0.0.0.0:5140"

But it doesn't look like the regex filter does:

# docker run --rm --name tinysyslog -p 2514:5140/udp admiralobvious/tinysyslog --log-level debug --filter-regex test --filter regex
time="2020-01-31T00:36:52Z" level=info msg="tinysyslog starting"
time="2020-01-31T00:36:52Z" level=debug msg="Using mutator type 'text'"
time="2020-01-31T00:36:52Z" level=debug msg="Using filter type 'regex' with filter ''"
time="2020-01-31T00:36:52Z" level=debug msg="Adding sink type 'console'"
time="2020-01-31T00:36:52Z" level=info msg="tinysyslog listening on 0.0.0.0:5140"

I'm not super sure, but it looks like maybe this line needs changed from filter-regex-filter to filter-regex?

https://github.com/admiralobvious/tinysyslog/blob/eae33b359fe126cbcc53635394f197fc326ec0ab/factories.go#L43

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.