Code Monkey home page Code Monkey logo

wufei's Introduction

WUFEI

Wufei is an async Rust CLI Tool for the aggregation of Kubernetes logs. This tool will write kubernetes logs for each pod down to a container level to a file or to stdout depending on the developers needs and also has the ability to log new pods that are spun up in the namespace as well. There is an informer written to let Wufei know when new pods spin up!

Heavily inspired by https://github.com/johanhaleby/kubetail Kubetail.

Wufei

Installation

As of right now, Wufei is NOT part of cargo. Its on my todo list. Right now just do cargo build in the root of the the project, and then access the wufei in target/debug/wufei

cargo run -- --namespace=<my-kube-namespace> --color

Example Output

Video coming soon

CLI Arguments

Wufei 0.3.1
Eric McBride <[email protected]> github.com/ericmcbride
Tail ALL your kubernetes logs at once, or record them to files

USAGE:
    wufei [FLAGS] [OPTIONS]

FLAGS:
        --color       Pods for the logs will appear in color in your terminal
    -f, --file        Record the logs to a file. Note: Logs will not appear in stdout
        --gather      Dont follow the logs, but gather all of them at once
    -h, --help        Prints help information
        --previous    Grab previous logs
        --update      Runs an informer, that will add new pods to the tailed logs
    -V, --version     Prints version information

OPTIONS:
        --json-key <json-key>        key to search for in the json, prints out the value. Only single key supported
    -n, --namespace <namespace>      Namespace for logs [default: kube-system]
    -o, --outfile <outfile>          Outfile of where the logs are being recorded [default: /tmp/wufei/]
        --selector <selector>        Select pods by label example: version=v1
        --since <since>              Only return logs newer then a duration in seconds like 1, 60, or 180
        --tail-lines <tail-lines>    If set, the number of lines from the end of the logs to show [default: 1]

Wufei requires a namespace.

  • The color flog --color will display pod names in colors in stdout.

  • The file flag --file will write the logs to /tmp/wufei/ based on pod name.

  • The update flag --update will spin up an informer that will listen for new pods to spin up

  • The previous flag --previous will show a previous containers logs. Specify --tail-lines or it will only show you the last line from it.

  • The --gather flag will gather all logs. To be used with --tail-lines or you will just get 1 log back.

  • The namespace option --namespace is the namespace the developer wants to use to tail logs from

  • The outfile option --outfile is used when the file flag is used, to change the location of where the files are used

  • The selector option --selector will allow a single key/value pair to tail logs by. Example would be --selector='version=v1'

  • The since option --since will return logs newer then the duration in seconds.

  • The tail-lines option --tail-lines will show the number of lines from the ends of the log to show. Defaults to 1

  • The json-key option --json-key allows the user to seach logs for a key in a valid json blob. The only thing wufei will print out are logs that contain the key. If nothing is printing out, nothing matches

Examples:

cargo run -- --namespace=default --color
cargo run -- --namespace=default --selector='version=v1' --update
cargo run -- --namespace=default --file --outfile=/tmp/new_outfile --update
cargo run -- --namespace=default --selector=`version=v1` --file
cargo run -- --namespace=default --previous --tail-lines=20 --color
cargo run -- --namespace=default --previous --color
cargo run -- --namespace=default --previous --since=1800
cargo run -- --namespace=default --previous --tail-lines=20 --color --selector='version=v1'
cargo run -- --namespace=default --color --json-key=X-REQUEST-ID
cargo run -- --namespace=default --file --json-key=user_id --select='version=v2'
cargo run -- --namespace=default --tail-lines=10000000 --gather --file

WUFEI USES YOUR CURRENT KUBE CONTEXT

export $KUBECONFIG=:$KUBECONFIG/path/to/your/config

LIST CONTEXTS

kubectl config view

USE CONTEXT

kubectl config use-context my-context

TOO MANY OPEN FILES ERROR

This error will pop up, depending on the settings on your operating system. This is due to security reasons. Below is how you would fix this on a Mac.

OS/X

ulimit -n 2048

Contributing

Pull requests are welcome. For major changes, please open an issue first to discuss what you would like to change.

Please make sure to update tests as appropriate.

License

MIT

wufei's People

Contributors

ericmcbride avatar treychadwell avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

wufei's Issues

Add Containers CLI Arg

Need to add a Containers CLI Arg that accepts multiples args. That we we can tail logs of specific containers instead of all of them.
--container foo --container bar
-c foo -c bar

Change informer to watch for modified kube deployment/pod files

We need to update the informer to see if a label or such has been added or modified during an edit, so we can pull it into the log streaming pool if theres a --selector argument.
1.) In the match statement under modified, see what we can key off of if a deployment/pod hais edited
2.) look up the pod that was edited, see if label matched what --selector key/value pair was passed in, if it was add to pool.

Add outfile flag

An outffile flag needs to be added to the cli to change the default path

Add since flag

Only return logs newer than a relative duration like 5s, 2m, or 3h

Refactor Code

Code could be simplified now that there is an outfile argument that defaults. Could probably do away with the generate_hashmap method. Also could probably swap to using actual rust kube bindings, and take advantage of the watcher API so we can update the logs if new pods are deployed.

Explore new Concurrency Methods

Write now wufei uses system threads. Maybe good to try to test out other methods of concurrency (futures, async, green threads via 3rd party lib)

Get rid of pods call in run_individual

I tried to call pods before run_individual and pass it in as a reference beforehand when collecting futures/tasks. I tried to hack around it and kept getting errors

   --> src/kube.rs:121:53
    |
117 |       let current_pods = Api::v1Pod(KubeClient::client().client.clone())
    |           ------------ move occurs because `current_pods` has type `kube::api::typed::Api<kube::api::resource::Object<k8s_openapi::v1_15::api::core::v1::pod_spec::PodSpec, k8s_openapi::v1_15::api::core::v1::pod_status::PodStatus>>`, which does not implement the `Copy` trait
...
121 |           children.push(tokio::task::spawn(async move {
    |  _____________________________________________________^
122 | |             run_individual(&pod, current_pods.clone()).await.unwrap()
    | |                                  ------------ use occurs due to use in generator
123 | |         }));
    | |_________^ value moved here, in previous iteration of loop

Add Selector flag

If selector label is added, pod name is ignored. Get all pods in a group

Add file writing functionality

Have base functionality for writing to a file already included. Keeping the file open and writing the logs until CTL + C is pressed

Add JSON parsing search option

Add a json parser using serde to record messages that match json strings/keys. This will allow for easy searching if you are looking for something in logs.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.