Code Monkey home page Code Monkey logo

jail_exporter's Introduction

Jail Exporter

Build Status

Description

Jail Exporter is a Prometheus exporter for FreeBSD jail metrics as reported by rctl(8).

The exporter is written in Rust and uses the jail and rctl crates to discover jails and obtain metrics.

Installation

pkg(8) and Ports

jail_exporter is available in the FreeBSD ports tree as sysutils/jail_exporter. It can be installed by either the binary package with pkg install jail_exporter, or by compiling the package yourself in the ports tree.

Cargo Install

The crate is also available via crates.io, and can be installed using the cargo install command. However, it is heavily recommended to install the exporter via pkg(8) or the ports tree.

$ cargo install jail_exporter

When installing via this method, you may want to move the installed binary to /usr/local/sbin and obtain an appropriate rc(8) to start the exporter on system boot. You can do this as follows:

# Performed as root.
$ mv ~/.cargo/bin/jail_exporter /usr/local/sbin/
$ chown root:wheel /usr/local/sbin/jail_exporter
$ jail_exporter --rc-script | tee /usr/local/etc/rc.d/jail_exporter
$ chmod 755 /usr/local/etc/rc.d/jail_exporter

Enabling at System Boot

You can enable jail_exporter to start at system boot via rc.conf(5).

# Either edit /etc/rc.conf directly or use the following command
sysrc jail_exporter_enable=YES

Building

At a minimum, building Jail Exporter should require:

  • Rust v1.70.0 (MSRV)
  • Cargo

A BSD make(1) Makefile is provided for convenience, if you already have Rust and Cargo available, executing make will build a debug release of Jail Exporter at target/debug/jail_exporter. make release will build a release version at target/release/jail_exporter.

If you don't wish to use make(1), the usual cargo build command should work just fine.

Configuration

Configuration can be performed either via command line arguments or environment variables. If both an environment variable and a command line argument are provided for the same option, the configuration will be taken from the command line argument.

Command Line Arguments

Argument Default Purpose
--output.file-path N/A Output metrics to a file instead of running an HTTPd.
--rc-script N/A Output an appropriate rc.d script
--web.auth-config N/A HTTP Basic authentication configuration file.
--web.listen-address 127.0.0.1:9452 Address on which to expose metrics and web interface.
--web.telemetry-path /metrics Path under which to expose metrics.

Environment variables

Variable Equivalent Argument
OUTPUT_FILE_PATH output.file-path
WEB_AUTH_CONFIG web.auth-config
WEB_LISTEN_ADDRESS web.listen-address
WEB_TELEMETRY_PATH web.telemetry-path

HTTP Basic Authentication

HTTP Basic Authentication is available when the crate is compiled with the auth feature. This feature is enabled by default.

The configuration file format is the same as specified in exporter-toolkit.

---
basic_auth_users:
    username: 'bcrypt hashed secret'
    another_user: 'bcrypt hashed secret'

If no configuration is specified, or the configuration is specified but contains no users, authentication will be disabled.

While loading the configuration, usernames will be checked for compliance with RFC7617. If any invalid usernames are detected, jail_exporter will error out and refuse to start.

User passwords are bcrypt hashed secrets, which can be generated with the bcrypt subcommand if jail_exporter was compiled with the bcrypt_cmd feature (enabled by default).

# Generate a password by specifying it on the CLI 
$ jail_exporter bcrypt foobar
Hash: $2b$12$kSOSps7NIQaEQi2CXb.Ll.qYQZ3pCK7NDWKUH7QvHzprUyLHL6tsq

# Generate a password via an interactive prompt
# The password will be prompted for and then confirmed
$ jail_exporter bcrypt
Password: [hidden]
Hash: $2b$12$/5ghhZFShmhrq5W4tE1hnubk/xgGH0MmixxQeKwSGx0g7kjDFL2hq

# Generate a random password
# The random plaintext password will also be output.
$ jail_exporter bcrypt --random
Password: TiFRz4rg6JHdRunnIFm2aB3uNa0OnlU7
Hash: $2b$12$dyK2iA1Yq1ToA9AWtjg96exmvLABj05DuB1V5a4haUOZvu2.Hvbo2

The cost for the password hashing can be controlled with the --cost argument to the bcrypt subcommand and defaults to 12. This default it taken from the bcrypt crate and should be tuned for the computer the hashing will be running on, as this cost will be paid each time a connection is authenticated.

Running

The exporter needs to run as root in order to have enough permission to execute the rctl_get_racct(2) calls. If it is not run as root, it will complain and exit.

As jails may come and go during the lifetime of the exporter, so to will the time series that the exporter exports. If you wish to account for resource usage for jails that have disappeared, you may wish to make use of the Prometheus recording rules to track total resource usage across all jails.

The exporter can be run in two different ways. The default way is to run a persistent network daemon for Prometheus to scrape. The exporter will not daemonize itself, instead, it is recommended to use a tool such as daemon(8). See the included rc.d/jail_exporter.in for an example of this.

The second way is to simply output the scraped metrics to a text file. This mode is designed to be paired with the node_exporter Textfile Collector.

No port is available yet, but it should happen soon.

Exposed Metrics

This exporter was developed under FreeBSD 11.1 and currently exports all resources listed under the RESOURCES section of rctl(8).

The id and num time series are calculated based on other information and do not come from rctl(8) directly. Metric names have their units appended where appropriate, based on the Prometheus best practice for metric and label naming.

All exported metrics are prefixed with jail and have a name label representing the name of the jail. As such, jail names are expected to be unique.

Descriptions of metrics are taken from the rctl(8) man page where applicable.

rctl(8) Metrics

Metric rctl(8) name Description
coredumpsize_bytes coredumpsize core dump size, in bytes
cputime_seconds_total cputime CPU time, in seconds
datasize_bytes datasize data size, in bytes
maxproc maxproc number of processes
memorylocked_bytes memorylocked locked memory, in bytes
memoryuse_bytes memoryuse resident set size, in bytes
msgqqueued msgqqueued number of queued SysV messages
msgqsize_bytes msgqsize SysV message queue size, in bytes
nmsgq nmsgq number of SysV message queues
nsem nsem number of SysV semaphores
nsemop nsemop number of SysV semaphores modified in a single semop(2) call
nshm nshm number of SysV shared memory segments
nthr nthr number of threads
openfiles openfiles file descriptor table size
pcpu_used pcpu %CPU, in percents of a single CPU core
pseudoterminals pseudoterminals number of PTYs
readbps readbps filesystem reads, in bytes per second
readiops readiops filesystem reads, in operations per second
shmsize_bytes shmsize SysV shared memory size, in bytes
stacksize_bytes stacksize stack size, in bytes
swapuse_bytes swapuse swap space that may be reserved or used, in bytes
vmemoryuse_bytes vmemoryuse address space limit, in bytes
wallclock_seconds_total wallclock wallclock time, in seconds
writebps writebps filesystem writes, in bytes per second
writeiops writeiops filesystem writes, in operations per second

Non-rctl(8) Metrics

Metric Description
exporter_build_info The version of Rust used to build the exporter, and the version of the exporter.
id ID of the named jail
num Current number of running jails

Crate Features

Feature Default Description
auth true Enables HTTP Basic Authentication
bcrypt_cmd true Enables a bcrypt subcommand to assist with hashing passwords for HTTP Basic Authentication
rc_script true Enables the --rc-script CLI flag to dump the rc(8) script to stdout

Notes

The rc_script feature is enabled by default for the benefit of users installing via cargo install. It is disabled by default in the FreeBSD port as the rc(8) script is supplied in the ports tree.

jail_exporter's People

Contributors

dependabot-preview[bot] avatar fabianfreyer avatar phyber avatar schueni1 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

jail_exporter's Issues

Split functionality up a bit more

Currently, everything lives in a main.rs. However, we should split this out into at least three different components.

  • main, the bit which parses command line args and calls other stuff.
  • httpd, the bit dealing with HTTP requests.
  • lib, the bit dealing with the actual gathering and exporting of metrics.

Work on this has already started in a branch.

Add {read,write}{bps,iops}

Although these are documented as being strange, we should still attempt to capture them in case people can make use of them in some way.

Add unit tests

After a few of the existing feature requests are in and stabilise a bit, unit testing would be nice.

Document installation in README

After the port is committed (#6), an Installation section should be added to the README.md detailing installation from ports/pkg and crates.io.

Update to a actix-web 1.0

As mentioned in subject, time to update. A lot of things have changed, but it shouldn't be too difficult.

Text file output mode

It would be nice to support a text file output mode, primarily intended to be used with the node_exporter textfile directory.
This would be useful for people who would prefer to run Jail Exporter from cron in order to have fewer network daemons running.

Move to using libjail-rs

After becoming comfortable with the C FFI, it's probably time to move to using libjail-rs for the jail_get calls.

RUSTSEC-2021-0139: ansi_term is Unmaintained

ansi_term is Unmaintained

Details
Status unmaintained
Package ansi_term
Version 0.12.1
URL ogham/rust-ansi-term#72
Date 2021-08-18

The maintainer has adviced this crate is deprecated and will not
receive any maintenance.

The crate does not seem to have much dependencies and may or may not be ok to use as-is.

Last release seems to have been three years ago.

Possible Alternative(s)

The below list has not been vetted in any way and may or may not contain alternatives;

See advisory page for additional details.

HTTP basic auth

Implement a simple HTTP basic auth, to cover the case where an admin wishes to protect the metrics.

RUSTSEC-2020-0036: failure is officially deprecated/unmaintained

failure is officially deprecated/unmaintained

Details
Status unmaintained
Package failure
Version 0.1.8
URL rust-lang-deprecated/failure#347
Date 2020-05-02

The failure crate is officially end-of-life: it has been marked as deprecated
by the former maintainer, who has announced that there will be no updates or
maintenance work on it going forward.

The following are some suggested actively developed alternatives to switch to:

See advisory page for additional details.

Jails may come and go, but time series stick around

Over the uptime of the exporter, many jails may come and go. Currently, any jails that disappear will continue to be represented within the exporter until it is restarted. eg. Poudriere jails that are up temporarily while building will disappear after the build is finished, but continue to be represented in the metrics.

Is this desirable? I don't think so, but those metrics should be represented somehow after the jail disappears. I see two easy methods for this:

  • Ship some example Prometheus recording rules which could keep overall totals for resource usage.
  • Have the exporter itself keep an "overall" series of metrics.

There's also the possibility of having old metrics stick around in the exporter for a while, but eventually "timeout" and be cleared up.

Finally, we could just not care about jails that disappear at all. At the beginning of each metrics gathering run all time series in the exporter would be cleared out, removing old jails that had disappeared.

It may be possible to implement each of these approaches and allow the user to toggle between them if there are good use cases for each scenario.

Counter Mutex not working quite right

jail_exporter/src/main.rs

Lines 207 to 223 in f5a2a60

"cputime" => {
// Get the value from last time.
let mut old_value = JAIL_CPUTIME_SECONDS_OLD.lock().unwrap();
// Work out what our increase should be.
// If old_value < value, OS counter has continued to increment,
// otherwise it has reset.
let inc = match *old_value <= *value {
true => *value - *old_value,
false => *value,
};
// Update book keeping.
*old_value = *value;
JAIL_CPUTIME_SECONDS.with_label_values(&[&name]).inc_by(inc);
},

Although this code compiles and runs,

*old_value = *value;
does not seem to have the desired effect.

old_value appears to only be settable once, after which it can never be set again.

DEBUG 2018-06-23T22:19:00Z: jail_exporter: Old: 42, New: 5
DEBUG 2018-06-23T22:19:00Z: jail_exporter: Inc: 5
DEBUG 2018-06-23T22:19:00Z: jail_exporter: Old: 5, New: 5

<scrape interval passes>

DEBUG 2018-06-23T22:20:00Z: jail_exporter: Old: 42, New: 5
DEBUG 2018-06-23T22:20:00Z: jail_exporter: Inc: 5
DEBUG 2018-06-23T22:20:00Z: jail_exporter: Old: 5, New: 5

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.