Code Monkey home page Code Monkey logo

50shades's Introduction

50shades (of Graylog)

Latest version License CI Status

Log trail and query client written in Rust.

50shades interfaces with Graylog's and Elasticsearch's query APIs so that log message lookups can be performed from the command line. It supports storing logins in native OS keychains and following up on queries, so that logs can be viewed in a tail -f or journalctl -f manner. 50shades unterstands intuitive English expressions for timespans. Output can be controlled using Handlebars syntax.

Usage

50shades provides several subcommands which come with their own respective sets of options. Invoking the help subcommand on any of the other subcommands, or passing --help to any of the subcommands will print the respective help screen for that command. Invoking help or passing --help without a subcommand prints general help.

USAGE:
    50shades [OPTIONS] <SUBCOMMAND>

FLAGS:
    -h, --help       Prints help information
    -V, --version    Prints version information

OPTIONS:
    -c, --config <config>        Path to custom configuration file
    -n, --node <node>            Node to query [default: default]
    -t, --template <template>    Template to use for output [default: default]

SUBCOMMANDS:
    follow    Follows the tail of a query (like tail -f on a log file)
    help      Prints this message or the help of the given subcommand(s)
    init      Initializes the configuration file
    login     Stores new password for specified node
    query     Performs one-time query

Before any actual queries can be performed by either query or follow, 50shades needs to be supplied with a valid TOML configuration file and a matching table for the specified node (defaults to default), which has to consist of a url and a user.
A valid configuration file looks like this:

[nodes.default]
url = 'https://graylog.example.com/api'
user = 'admin'
type = 'graylog'

[nodes.elastic]
url = 'https://elastic.example.com/'
user = 'elastic'
type = 'elastic'

[nodes.logstash]
url = 'https://elastic.example.com/logstash-*'
user = 'elastic'
type = 'elastic'

[nodes.elastic-noauth]
url = 'https://elastic.example.com/'
type = 'elastic'

[templates]
default = '[{{default container_name "-"}}] {{{message}}}'
rocket = '{{{method}}}{{{route}}} {{{uri}}}{{{status}}}'

Here, 50shades invocations without a node specified would attempt to query the Graylog server API at https://graylog.example.com/api with the user admin. By specifying -n elastic, it would instead query the Elasticsearch server at https://elastic.example.com/ for all indices and attempt to authenticate the user elastic. Specifying -n logstash would limit the same queries against indices starting in logstash-, whereas -n elastic-noauth would query all indices, but not attempt any authentication, which is a viable option for Elasticsearch, but not for Graylog.

In addition, a matching password has to be stored for a node if a username is specified. This can be done by invoking 50shades with the login command while specifying the desired node using -n to store the password for.

Any additional query or follow arguments after the options are passed down to Graylog or Elasticsearch as the actual query and use Lucene query syntax, just like they do in the respective tools.

Default Configuration File

The location of the default configuration file is operating system dependent. To have it created with sensible values and learn about its location, 50shades provides the init command which prompts for a url, user name and password and prints the path to the file. Initializing the configuration file also writes out the default output templates which is further explained below.

Controlling Output

Each query result is output as a single line, controlled by the Handlebars template referenced by the --template, or -t, option. 50shades' default template is specified as follows:

[{{default container_name "-"}}] {{{message}}}

50shades includes default as a custom helper which may be used to specify a default value if a field is missing in a query result. Otherwise, an empty string would be generated.

Password Storage

50shades supports reading passwords from operating system / desktop environment keyrings, only. Passwords cannot be stored in configuration nor passed or piped during invocation.

Installation

The easiest way to install 50shades is by having a working Rust toolchain installed and invoking

cargo install fifty-shades

which will place the resulting binary in ~/.cargo/bin.

Copyright

Copyright 2019 Communicatio.Systems GmbH

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

50shades's People

Contributors

outergod avatar

Stargazers

 avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

Forkers

kuhmuh ghostolli

50shades's Issues

Use of templates

Hi,
I don't manage to get the templates working... Could you elaborate a bit more on this? I ha trying to get a template that shows the logs for haproxy instances and it looks like:

haproxy = '[{{default source "haproxy01.domain.com"}}] {{message}}'

So I need logs where source:haproxy01.domain.com
Is it also possible to use them without having to define them in config.toml?

Cheers.

Timestamp parsing issues

I am trying to filter by timestamp but it seems there are parsing issues:

50shades  follow   timestamp:["2019-07-23 09:53:08.175" TO "2019-07-23 09:53:08.575"]
Error: 500 Internal Server Error: Unable to perform search query

parse_exception: Encountered " <RANGE_GOOP> "09:53:08.175 "" at line 1, column 22.
Was expecting:
    "TO" ...

no Keyring in the root during `cargo install`

this looks like an awesome tool! Is it possible to make keyring optional, for platforms that don't support it? or is there a .so missing that it requires?

   Compiling fifty-shades v0.2.0
     Running `rustc --edition=2018 --crate-name 50shades /home/dch/.cargo/registry/src/github.com-1ecc6299db9ec823/fifty-shades-0.2.0/src/main.rs --error-format=json --json=diagnostic-rendered-ansi --crate-type bin --emit=dep-info,link -C opt-level=3 -C metadata=b2b6e294ea37cd61 -C extra-filename=-b2b6e294ea37cd61 --out-dir /tmp/cargo-installHHvfcQ/release/deps -L dependency=/tmp/cargo-installHHvfcQ/release/deps --extern chrono=/tmp/cargo-installHHvfcQ/release/deps/libchrono-0c8a093d1dd38cda.rlib --extern dialoguer=/tmp/cargo-installHHvfcQ/release/deps/libdialoguer-021ae39553ba0fc1.rlib --extern dirs=/tmp/cargo-installHHvfcQ/release/deps/libdirs-306d3cef961caaf2.rlib --extern exitfailure=/tmp/cargo-installHHvfcQ/release/deps/libexitfailure-ddea1353f3592ba6.rlib --extern failure=/tmp/cargo-installHHvfcQ/release/deps/libfailure-128028cd3192fa66.rlib --extern handlebars=/tmp/cargo-installHHvfcQ/release/deps/libhandlebars-6b3fb443248ba40f.rlib --extern keyring=/tmp/cargo-installHHvfcQ/release/deps/libkeyring-2be0292762a6a177.rlib --extern maplit=/tmp/cargo-installHHvfcQ/release/deps/libmaplit-df591f7b99666181.rlib --extern reqwest=/tmp/cargo-installHHvfcQ/release/deps/libreqwest-6d883f08fa987129.rlib --extern serde=/tmp/cargo-installHHvfcQ/release/deps/libserde-556ba56baa19e94e.rlib --extern serde_json=/tmp/cargo-installHHvfcQ/release/deps/libserde_json-f072ef80b1aa4921.rlib --extern structopt=/tmp/cargo-installHHvfcQ/release/deps/libstructopt-2a6b99e00276dfa8.rlib --extern toml=/tmp/cargo-installHHvfcQ/release/deps/libtoml-28f56bc5f054062d.rlib --extern two_timer=/tmp/cargo-installHHvfcQ/release/deps/libtwo_timer-2e53a484aaab712a.rlib --extern url=/tmp/cargo-installHHvfcQ/release/deps/liburl-804eb20f3fc69d55.rlib --cap-lints allow -L native=/tmp/cargo-installHHvfcQ/release/build/backtrace-sys-ab068673c85ef8d9/out -L native=/usr/lib`
error[E0432]: unresolved import `keyring::Keyring`
  --> /home/dch/.cargo/registry/src/github.com-1ecc6299db9ec823/fifty-shades-0.2.0/src/password.rs:19:15
   |
19 | use keyring::{Keyring, KeyringError};
   |               ^^^^^^^ no `Keyring` in the root

error: aborting due to previous error

For more information about this error, try `rustc --explain E0432`.
error: failed to compile `fifty-shades v0.2.0`, intermediate artifacts can be found at `/tmp/cargo-installHHvfcQ`

Caused by:
  could not compile `fifty-shades`.

Caused by:
  process didn't exit successfully: `rustc --edition=2018 --crate-name 50shades /home/dch/.cargo/registry/src/github.com-1ecc6299db9ec823/fifty-shades-0.2.0/src/main.rs --error-format=json --json=diagnostic-rendered-ansi --crate-type bin --emit=dep-info,link -C opt-level=3 -C metadata=b2b6e294ea37cd61 -C extra-filename=-b2b6e294ea37cd61 --out-dir /tmp/cargo-installHHvfcQ/release/deps -L dependency=/tmp/cargo-installHHvfcQ/release/deps --extern chrono=/tmp/cargo-installHHvfcQ/release/deps/libchrono-0c8a093d1dd38cda.rlib --extern dialoguer=/tmp/cargo-installHHvfcQ/release/deps/libdialoguer-021ae39553ba0fc1.rlib --extern dirs=/tmp/cargo-installHHvfcQ/release/deps/libdirs-306d3cef961caaf2.rlib --extern exitfailure=/tmp/cargo-installHHvfcQ/release/deps/libexitfailure-ddea1353f3592ba6.rlib --extern failure=/tmp/cargo-installHHvfcQ/release/deps/libfailure-128028cd3192fa66.rlib --extern handlebars=/tmp/cargo-installHHvfcQ/release/deps/libhandlebars-6b3fb443248ba40f.rlib --extern keyring=/tmp/cargo-installHHvfcQ/release/deps/libkeyring-2be0292762a6a177.rlib --extern maplit=/tmp/cargo-installHHvfcQ/release/deps/libmaplit-df591f7b99666181.rlib --extern reqwest=/tmp/cargo-installHHvfcQ/release/deps/libreqwest-6d883f08fa987129.rlib --extern serde=/tmp/cargo-installHHvfcQ/release/deps/libserde-556ba56baa19e94e.rlib --extern serde_json=/tmp/cargo-installHHvfcQ/release/deps/libserde_json-f072ef80b1aa4921.rlib --extern structopt=/tmp/cargo-installHHvfcQ/release/deps/libstructopt-2a6b99e00276dfa8.rlib --extern toml=/tmp/cargo-installHHvfcQ/release/deps/libtoml-28f56bc5f054062d.rlib --extern two_timer=/tmp/cargo-installHHvfcQ/release/deps/libtwo_timer-2e53a484aaab712a.rlib --extern url=/tmp/cargo-installHHvfcQ/release/deps/liburl-804eb20f3fc69d55.rlib --cap-lints allow -L native=/tmp/cargo-installHHvfcQ/release/build/backtrace-sys-ab068673c85ef8d9/out -L native=/usr/lib` (exit code: 1)

Graylog limits the number of output items to 150

No matter what the query or timespan is, the maximum number of result items returned is always 150. According to Graylog devs, to get more result items, one has to set the limit query parameter accordingly or page through the result set (the offset query parameter). Would be nice to have this implemented.

Special characters not properly displayed

Hi,
I see that logs from Graylog are fine, but when I use the tool I see "&quot" and those sort of characters.

[k1n01.domain.com] time=&quot;2019-10-18T10:08:15Z&quot; level=debug msg=&quot;

MacOS builds

Hi,
would it be possible to have the MacOS packages also available? I tried the cross-compile option but it didn't work.
Thanks.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.