Code Monkey home page Code Monkey logo

scriptablebeat's Introduction

Scriptable Beat

Status Docker image size Docker pulls

A Beat that allows the user to run any arbitrary script inside of it, to do the collection of the data, and then takes care of transmitting the output to the LogRhythm Open Collector, LogRhythm System Monitor Agent or LogStash over Lumberjack.

Key constrains and goals

  • Run Python script
  • Run PowerShell script
  • Run Bash script
  • Manage the scheduling of the script run
  • Monitor the script output and push it to the OC/SMA
  • Deal gracefully with errors/crashs of the script
  • Manage the configuration of the script through its own configuration
  • Ideally - Manage/clean the script output from the disk, preventing any unnecessary disk usage
  • Ideally - Deal with (Lumberjack) back-pressure by reducing/stopping scheduling frequency

To Deploy

Visit the Releases page.

High level architecture

Container containing

  • Beat core code
  • Interpreters for the selected supported scripts

Configuration

  • What packages/modules are required for the scripts
  • What langage/interpreter is to be used
  • Content of the:
    • First Run script
    • Startup script
    • Scheduled script
  • Scheduler details:
    • Run once and keep running?
    • Frequency?
    • Restart on Error?
    • Restart on Crash?
  • Dictate if the log data is meant to be coming out of STDOUT, STDERR, file/folder or any mix and match
  • Dictate which script needs to be monitored
    • First Run script
    • Startup script
    • Scheduled script
  • File/Folder path
  • How to treat output:
    • JSON log as-is
    • Plain text that needs to be encoded into a JSON field
  • How to handle STDERR
    • Report as log message in the stream
    • Report as error in the stream
    • Log in Beats own logs
    • All of the above
    • Ignore
  • How to handle STDOUT
    • Report as log message in the stream
    • Report as error in the stream
    • Log in Beats own logs
    • All of the above
    • Ignore
  • How to handle files' content
    • Report as log message in the stream
    • Report as error in the stream
    • Log in Beats own logs
    • All of the above
    • Ignore
  • How to handle File after read/processing
    • Leave as is
    • Delete
    • Flush to empty
  • How to handle Folder left empty
    • Leave as is
    • Delete

At startup

  • Start run sequence (only passing to next step on success of each step):
    • Read the configuration file
    • Establish comms with OC/SMA and other internal Beat required prep tasks
    • Enable Heartbeat
    • Read from the configuration the list of required packages/modules required by the script
    • Download/update each of the said packages/modules
    • Run First Run script (only once, at the very first startup)
    • Run Startup script (at each startup)
    • Run Scheduler

At run time

Beat core to monitor:

  • STDOUT
  • STDERR
  • File/folder specified in configuration

Internal doc

https://logrhythm.atlassian.net/l/cp/kmMun7qV

scriptablebeat's People

Contributors

tonymasse avatar

Stargazers

Rochelle Lewis avatar

Watchers

 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.