Code Monkey home page Code Monkey logo

boxspy's Introduction

BoxSpy

GroundWork BoxSpy

This project is a fork of cAdvisor v 0.5.0 to provide a trimmed down version of a monitor container. The purpose is to provide a small footprint container to collect stats monitoring statistics for all containers running on the docker engine. The goal is to optimze the data collection and transfer (API based) Managment systems.

This is experimental and the goal is to submit the boxer settings upstream as a set of runtime switches. We envision two modes:

  • Operation/Production mode (boxer settings) that are optimized far large deployments and long running containers
  • Developer mode (cAdvisor) for monitoring local deployments, rapid development and troubleshooting in DevOps environments

Quick Start: Running BoxSpy in a Docker Container

To quickly tryout BoxSpy on your machine with Docker (version 1.3 or above), we have a Docker image that includes everything you need to get started. Simply run:

sudo docker run \
  --volume=/:/rootfs:ro \
  --volume=/var/run:/var/run:rw \
  --volume=/sys:/sys:ro \
  --volume=/var/lib/docker/:/var/lib/docker:ro \
  --publish=8081:8080 \
  --detach=true \
  --name=BoxSpy \
  gwos/boxspy:canary

Please note that the container will run on port 8081 and therefore the UI is available through: http://localhost:8081

Remote REST API

Current version is v1.2 Example to get all docker instances: http://localhost:8081/api/v1.2/docker

Notes from the original project from which BoxSpy was forked

cAdvisor

cAdvisor (Container Advisor) provides container users an understanding of the resource usage and performance characteristics of their running containers. It is a running daemon that collects, aggregates, processes, and exports information about running containers. Specifically, for each container it keeps resource isolation parameters, historical resource usage, histograms of complete historical resource usage and network statistics. This data is exported by container and machine-wide.

cAdvisor currently supports lmctfy containers as well as Docker containers (those that use the default libcontainer execdriver). Other container backends can also be added. cAdvisor's container abstraction is based on lmctfy's so containers are inherently nested hierarchically.

cAdvisor

Quick Start: Running cAdvisor in a Docker Container

To quickly tryout cAdvisor on your machine with Docker (version 0.11 or above), we have a Docker image that includes everything you need to get started. Simply run:

sudo docker run \
  --volume=/:/rootfs:ro \
  --volume=/var/run:/var/run:rw \
  --volume=/sys:/sys:ro \
  --volume=/var/lib/docker/:/var/lib/docker:ro \
  --publish=8080:8080 \
  --detach=true \
  --name=cadvisor \
  google/cadvisor:latest

cAdvisor is now running (in the background) on http://localhost:8080. The setup includes directories with Docker state cAdvisor needs to observe.

Note: On CentOS and RHEL 6, run cAdvisor with an additional option --volume=/cgroup:/cgroup \

If you want to build your own cAdvisor Docker image, take a look at deploy/Dockerfile and deploy/build.sh.

Using InfluxDB as backend storage

cAdvisor now also supports InfluxDB to store stats. To use InfluxDB, you need to pass some additional flags to cAdvisor:

Required

 # storage driver to use. Options are: memory (default) and influxdb
 -storage_driver=influxdb
 # Required to make glog work
 -log_dir=/

Optional

 # The *ip:port* of the database. Default is 'localhost:8086'
 -storage_driver_host=ip:port
 # database name. Uses db 'cadvisor' by default
 -storage_driver_name
 # database username. Default is 'root'
 -storage_driver_user
 # database password. Default is 'root'
 -storage_driver_password
 # Use secure connection with database. False by default
 -storage_driver_secure

Cluster monitoring using cAdvisor

Heapster enables cluster wide monitoring of containers using cAdvisor.

Web UI

cAdvisor exposes a web UI at its port:

http://<hostname>:<port>/

Remote REST API

cAdvisor exposes its raw and processed stats via a versioned remote REST API:

http://<hostname>:<port>/api/<version>/<request>

The current version of the API is v1.2.

Version 1.2

This version exposes the same endpoints as v1.1 with one additional read-only endpoint.

Docker Container Information

The resource name for Docker container information is as follows:

/api/v1.1/docker/<Docker container name or blank for all Docker containers>

The Docker name can be either the UUID or the short name of the container. It returns the information of the specified container(s). The information is returned as a list of serialized ContainerInfo JSON objects (found in info/container.go).

Version 1.1

This version exposes the same endpoints as v1.0 with one additional read-only endpoint.

Subcontainer Information

The resource name for subcontainer information is as follows:

/api/v1.1/subcontainers/<absolute container name>

Where the absolute container name follows the lmctfy naming convention (described bellow). It returns the information of the specified container and all subcontainers (recursively). The information is returned as a list of serialized ContainerInfo JSON objects (found in info/container.go).

Version 1.0

This version exposes two main endpoints, one for container information and the other for machine information. Both endpoints are read-only in v1.0.

Container Information

The resource name for container information is as follows:

/api/v1.0/containers/<absolute container name>

Where the absolute container name follows the lmctfy naming convention. For example:

Container Name Resource Name
/ /api/v1.0/containers/
/foo /api/v1.0/containers/foo
/docker/2c4dee605d22 /api/v1.0/containers/docker/2c4dee605d22

Note that the root container (/) contains usage for the entire machine. All Docker containers are listed under /docker.

The container information is returned as a JSON object containing:

  • Absolute container name
  • List of subcontainers
  • ContainerSpec which describes the resource isolation enabled in the container
  • Detailed resource usage statistics of the container for the last N seconds (N is globally configurable in cAdvisor)
  • Histogram of resource usage from the creation of the container

The actual object is the marshalled JSON of the ContainerInfo struct found in info/container.go

Machine Information

The resource name for machine information is as follows:

/api/v1.0/machine

This resource is read-only. The machine information is returned as a JSON object containing:

  • Number of schedulable logical CPU cores
  • Memory capacity (in bytes)

The actual object is the marshalled JSON of the MachineInfo struct found in info/machine.go

REST API Clients

There is an example Go client under client/ - you can use it on your own Go project by including it like this:

import "github.com/gwos/boxspy/client"
client, err = client.NewClient("http://192.168.59.103:8080/")
mInfo, err := client.MachineInfo()

Roadmap

cAdvisor aims to improve the resource usage and performance characteristics of running containers. Today, we gather and expose this information to users. In our roadmap:

  • Advise on the performance of a container (e.g.: when it is being negatively affected by another, when it is not receiving the resources it requires, etc)
  • Auto-tune the performance of the container based on previous advise.
  • Provide usage prediction to cluster schedulers and orchestration layers.

Community

Contributions, questions, and comments are all welcomed and encouraged! cAdvisor developers hang out in #google-containers room on freenode.net. We also have the google-containers Google Groups mailing list.

boxspy's People

Contributors

ashahab-altiscale avatar blakelapierre avatar caglar10ur avatar cburroughs avatar danmcp avatar dchen1107 avatar gwos-eng avatar highlyunavailable avatar kelseyhightower avatar kevin1024 avatar kyurtsever avatar monnand avatar pborreli avatar plietar avatar proppy avatar rjnagal avatar satnam6502 avatar vishh avatar vmarmol avatar zohaib1020 avatar

Watchers

 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.