Code Monkey home page Code Monkey logo

kubesec's Introduction

kubesec Latest Version Build Status

Secure secret management for Kubernetes (with gpg, Google Cloud KMS and AWS KMS backends).

asciicast

In short, it allows you to encrypt Secrets so that they can be stored in VCS along with the rest of resources.
An example of encrypted Secret is shown below (note that only the "data" is encrypted (and that keys are left untouched)):

apiVersion: v1
kind: Secret
metadata:
  name: myapp-default-0
type: Opaque
data:
  KEY: TUFkWD1iuKs=.O....D...=
  ANOTHER_KEY: iOy1nf90+M6FrrEIoymN6cOSUYM=.E...=.q...=
# ...  

The nice thing about this approach (compared to complete file encryption) is that git diff and git merge become so much more user-friendly (+ you can ascertain that specific entry is present even if you don't have the key to decrypt the secret).

kubesec is written in Go, works with (or without) Yubikey ❤.

For general-purpose secret management, take a look at mozilla/sops
(kubesec's drawn a lot of inspiration from it).

Installation

macOS

curl -sSL https://github.com/shyiko/kubesec/releases/download/0.9.2/kubesec-0.9.2-darwin-amd64 \
  -o kubesec && chmod a+x kubesec && sudo mv kubesec /usr/local/bin/  

Verify PGP signature (optional but recommended):

curl -sSL https://github.com/shyiko/kubesec/releases/download/0.9.2/kubesec-0.9.2-darwin-amd64.asc \
  -o kubesec.asc
curl -sS https://keybase.io/shyiko/pgp_keys.asc | gpg --import
gpg --verify kubesec.asc /usr/local/bin/kubesec

gpg can be installed with brew install gnupg

... with Homebrew
brew install shyiko/kubesec/kubesec
brew install shyiko/kubesec/kubesec --with-short-name # install as "ksec"

brew install shyiko/kubesec/kubesec is equivalent to brew tap shyiko/kubesec && brew install kubesec.

Linux

curl -sSL https://github.com/shyiko/kubesec/releases/download/0.9.2/kubesec-0.9.2-linux-amd64 \
  -o kubesec && chmod a+x kubesec && sudo mv kubesec /usr/local/bin/  

Verify PGP signature (optional but recommended):

curl -sSL https://github.com/shyiko/kubesec/releases/download/0.9.2/kubesec-0.9.2-linux-amd64.asc \
  -o kubesec.asc
curl -sS https://keybase.io/shyiko/pgp_keys.asc | gpg --import
gpg --verify kubesec.asc /usr/local/bin/kubesec

Windows

Download executable from the Releases page.

Usage

If you plan to use gpg:
... but don't have a valid PGP key, see GitHub Help - Generating a new GPG key on how to generate one.
gpg (tested: 2.0+; recommended: 2.1+) must be available on the PATH.
It's also highly recommended to set up gpg-agent to avoid constant passphrase re-entry.

Encryption

# encrypt existing Secret (see `kubesec create` below on how to create encrypted secret from scratch)
kubesec encrypt secret.yml
# same as above but output is written back to secret.yml (instead of stdout)
kubesec encrypt -i secret.yml

# NOTE: if you don't specify --key - default PGP key will be used
# in other words, `kubesec encrypt secret.yml` is identical to 
kubesec encrypt --key=pgp:default secret.yml

# NOTE: multiple --key|s can be specified if needed 
# (and they don't have to be of the same type, i.e. `--key=pgp:... --key=arn:...` 
# is perfectly valid)

# encrypt with PGP key ("pgp:" prefix is optional)
kubesec encrypt --key=pgp:6206C32E111611688694CF5530BDA87E3E71C268 secret.yml

# avoid gpgagent for pgp passprhase when encrypting with PGP key ("pgp:" prefix is optional)
kubesec encrypt --passphrase=<supersecret> --key=pgp:6206C32E111611688694CF5530BDA87E3E71C268 secret.yml

# encrypt with Google Cloud KMS key ("gcp:" prefix is optional)
#
# NOTE: you'll need either to `gcloud auth application-default login` or set
# GOOGLE_APPLICATION_CREDENTIALS=/path/to/credentials.json 
# before attempting secret encryption/decryption
#
# https://developers.google.com/identity/protocols/application-default-credentials#howtheywork
kubesec encrypt --key=gcp:<resource-id of Google Cloud KMS key> secret.yml
kubesec encrypt \ 
  --key=gcp:projects/project-0/locations/global/keyRings/keyring-0/cryptoKeys/key-0 secret.yml

# encrypt with AWS KMS key ("aws:" prefix is optional)
#
# NOTE: you might need to `aws configure` (if you don't have ~/.aws/credentials already)
#
# http://docs.aws.amazon.com/cli/latest/userguide/cli-chap-getting-started.html
kubesec encrypt --key=aws:<ARN of AWS KMS key> secret.yml
kubesec encrypt \
  --key=aws:arn:aws:kms:us-west-1:000000000000:key/00000000-0000-0000-0000-000000000000 secret.yml

# add ...D89 key & drop ...310 key (leave all other keys untouched)
kubesec encrypt --key=+pgp:160A7A9CF46221A56B06AD64461A804F2609FD89 \
  --key=-pgp:6206C32E111611688694CF5530BDA87E3E71C268 secret.yml
# NOTE: removal of a key will automatically result in data encryption key rotation
# you will also need to change all the secrets as whoever you removed from the chain of trust might 
# still have access to the previous version of a file   

# encrypt content of stdin
cat secret.yml | kubesec encrypt -

# create encrypted Secret from key=value pair(s) / file(s)
kubesec create secret-name \
  --data key=value \
  --data file:pki/ca.crt \
  --data file:hostname.key=pki/private/server.key \
  -o secret.enc.yml

Decryption

# decrypt a Secret 
# (usually combined with kubectl (`kubesec decrypt secret.enc.yml | kubectl apply -f -`))
kubesec decrypt secret.enc.yml 

# decrypt without invoking gpgagent - useful for unattended interaction or for alternate keyrings 
#
# You can prevent these lines ending up in history with a space 
# see https://www.linuxjournal.com/content/using-bash-history-more-efficiently-histcontrol or https://superuser.com/questions/352788/how-to-prevent-a-command-in-the-zshell-from-being-saved-into-history 
kubesec decrypt --keyring alternate.keyring --passphrase=<supersecret> secret.yml

# decrypt to a custom Go Template (http://golang.org/pkg/text/template) string
kubesec decrypt secret.enc.yml --cleartext --template='KEY={{ .data.KEY }}'
kubesec decrypt secret.enc.yml --cleartext \
  --template=$'{{ range $k, $v := .data }}{{ $k }}={{ $v }}\n{{ end }}' > .env

Modification

# open decrypted Secret in $EDITOR (it will be automatically re-encrypted upon save)
kubesec edit -i secret.enc.yml
kubesec edit -i --key=<a-different-key-to-re-encrypt-with> secret.enc.yml
# same as above but secret.enc.yml will be created if it doesn't exist 
kubesec edit -if secret.enc.yml

# batch editing (noninteractive)
kubesec patch -i secret.enc.yml --data key1=secret_string --data file:key2=path/to/file

# "decrypt, modify-in-any-way-you-like, re-encrypt" 
kubesec decrypt --cleartext secret.enc.yml -o secret.yml
# edit secret.yml using your favourite editor / tool
kubesec encrypt --cleartext secret.yml -o secret.enc.yml --parent=secret.enc.yml
# --parent=path/to/encrypted/secret.enc.yml above is used to preserve keys, DEK and IVs (when safe)

Introspection

# show information about the Secret (who has access to the "data", last modification date, etc)
kubesec introspect secret.enc.yml

Tab completion

# bash
source <(kubesec completion bash)

# zsh
source <(kubesec completion zsh)

- can be used anywhere (where a file is expected) to reference stdin.
(for more information see kubesec --help)

Example(s)

#1 (basic)

kubesec create secret-name -d key=value -d file:path/to/file -o secret.enc.yml  
kubesec decrypt secret.enc.yml | kubectl apply -f -

Playground

If you have docker installed you don't need to download kubesec binary just to try it out.
Instead, launch a container and start playing:

docker run -it --rm shyiko/kubesec-playground:0.9.2 /bin/bash
$ kubesec encrypt secret.yml

shyiko/kubesec-playground image contains gpg 2.1+, kubesec, vim (as a default $EDITOR) and secret PGP key of Jean-Luc Picard (PGP fingerprint - 6206C32E111611688694CF5530BDA87E3E71C268).

Dockerfile is included within this repo.

Encryption Protocol

  • "data" values are encrypted with AES-GCM (each value is padded to a block-size (48 bytes by default) and then encrypted using a shared (resource-unique, randomly generated) 256-bit DEK & a 96-bit random IV).
  • DEK is encrypted (and signed in case of PGP) with --key(s) before being stored in a Secret as # kubesec:<key type>:<key id>:... (one entry for each --key).

In addition to the above, kubesec also generates MAC (AES-GMAC, with AAD constructed from both the "data" and the --key(s)). If MAC is missing or invalid - decryption will fail (kubesec edit -i --recompute-mac <file> can be used to recompute MAC when necessary (e.g. after git merge)).

Reporting Security Issues

Please reach me at https://keybase.io/shyiko.

Development

PREREQUISITE: go1.9+.

git clone https://github.com/shyiko/kubesec $GOPATH/src/github.com/shyiko/kubesec 
cd $GOPATH/src/github.com/shyiko/kubesec
make fetch

go run kubesec.go

Legal

All code, unless specified otherwise, is licensed under the Apache-2.0 license.
Copyright (c) 2018 Stanley Shyiko.

kubesec's People

Contributors

prestonvanloon avatar shyiko avatar stickycode avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

kubesec's Issues

secret was encrypted with newer version error

I use the latest version of kubesec but see this message when decrypting a secret that I encrypted previously on another machine:

kubesec decrypt secret-oxidixed-id-rsa.pub.yml
It appears that Secret was encrypted with newer version of kubesec.
Visit https://github.com/shyiko/kubesec for upgrade instructions.

I'm using the latest version though:

kubesec --version
0.9.2

I'm using AWS KMS. Any idea how to troubleshoot this?

`gpg` wasn't found (make sure it's available on the PATH)

Hi,

I'm trying kubesec on my Macbook, and I got this error :

`gpg` wasn't found (make sure it's available on the PATH)

But, it's strange, because gpg command work fine :

15:05:18 › which gpg
/usr/local/bin/gpg

15:05:20 › gpg
gpg: Go ahead and type your message ...
^C
gpg: Interrupt caught ... exiting

Thanks a lot, this tool look awesome.

Decryption with multiple PGP keys fails

This is a question for better understanding:

When adding multiple PGP keys to a secret, it seems as if you can only decrypt the secret when also having the pubkey of the person who added your key:

I added a colleague key, however he wasnt able to decrypt until he imported my pubkey to his keyring.

This is the debug output

 kubesec patch foobar.yaml --debug
Executing sh -c "/usr/local/MacGPG2/bin/gpg2 --list-secret-keys --with-colons --fingerprint --fingerprint" 
Attempting to decrypt DEK with PGP key _HIS_KEY_ID_ 
Executing sh -c "/usr/local/MacGPG2/bin/gpg2 -d --status-fd 3 -o /var/folders/3s/y699yyld6mj8h32yt68nblt40000gn/T/808465590E /var/folders/3s/y699yyld6mj8h32yt68nblt40000gn/T/808465590" 
gpg: encrypted with 4096-Bit RSA key, ID  _HIS_KEY_ID_ ,..
gpg: Signature made Thu Nov ..
gpg: using RSA key _MY_KEY_ID
gpg: signature verification failed: No public key
Unable to decrypt DEK with PGP key _HIS_KEY_ID_  (Signature is invalid or missing) 
Unable to decrypt Data Encryption Key (DEK) 

This behavior is rather unfavorable as every decrypter needs to retrieve and import all other pubkeys.

Primary PGP key wasn't found

Hi,

I'm trying to create a secret on my Mac but i always get this error even though i've created my GPG key:

Primary PGP key wasn't found

Thanks.

do not support new mac OS version on 12.0.1(21A559)

@shyiko
how can I resolve it ?

$  kubesec
fatal error: runtime: bsdthread_register error
runtime stack:
runtime.throw(0x156bf2a, 0x21)
        /home/shyiko/Development/golang/go1.9.1.linux-amd64/src/runtime/panic.go:605 +0x95 fp=0x7ff7bfeff110 sp=0x7ff7bfeff0f0 pc=0x1029ad5
runtime.goenvs()
        /home/shyiko/Development/golang/go1.9.1.linux-amd64/src/runtime/os_darwin.go:108 +0x83 fp=0x7ff7bfeff140 sp=0x7ff7bfeff110 pc=0x1027373
runtime.schedinit()
        /home/shyiko/Development/golang/go1.9.1.linux-amd64/src/runtime/proc.go:482 +0xa1 fp=0x7ff7bfeff180 sp=0x7ff7bfeff140 pc=0x102c481
runtime.rt0_go(0x7ff7bfeff1b0, 0x1, 0x7ff7bfeff1b0, 0x1000000, 0x1, 0x7ff7bfeff3c8, 0x0, 0x7ff7bfeff3d0, 0x7ff7bfeff3e2, 0x7ff7bfeff3f6, ...)
        /home/shyiko/Development/golang/go1.9.1.linux-amd64/src/runtime/asm_amd64.s:175 +0x1eb fp=0x7ff7bfeff188 sp=0x7ff7bfeff180 pc=0x1053d4b

Kubesec should generate valid secrets with the cleartext flag

Sometime around 1.6, I believe, the .stringData field was added to the Secret spec, allowing non-b64-encoded data to be supplied to the apiserver. This is quite handy for managing updates with secrets without having to go through the whole decoding/encoding cycle.

Kubeseal already supports decoding base64 secrets, but outputs the raw strings to .data, which makes for invalid secrets; outputting that to .stringData would retain validity.

kubesec edit -if fails

The README suggests that it would create a new file and encrypt it on save. However the command fails with

kubesec edit -if new-secret.yaml
Failed to base64-decode _

Space(s) just before LF makes encrypted data mess

If there is/are space(s) just before a line feed in file, encrypted data will be messed.
When you open the encrypted file with kubesec edit, you will see encoded line feeds like \n.
I'm using 0.9.2 with Google Cloud KMS.

Flag to avoid using gpg agent

Passing --batch --no-pinentry --passphrase="something" (or prompt without) to gpg is useful when you are doing testing and validation locally and don't want to mess with your default keyring.

I'd like a --no-pinentry flag that or a --gpg-flags flag to pass things on to gpg under the hood

Github documentation about gpg key generation leads to "Unable to decrypt Data Encryption Key (DEK)"

Hi,

I'm working to use this great tools from within GItlab CI with GPG Key. I follow the documentation link in the readme about GitHub Help - Generating a new GPG key.

But, if I follow it correctly, I have the following results:

gpg --list-secret-key
/root/.gnupg/pubring.kbx
------------------------
sec   rsa4096 2019-08-13 [SC]
      ABE7E706C03B05B9E65AB3C9BCFF30F7C2D1589E
uid           [ultimate] gitlab-ci <[email protected]>
ssb   rsa4096 2019-08-13 [E]

If I try to use kubesec, I have the following result:

kubesec encrypt secret.yml --debug
Executing sh -c "/usr/bin/gpg2 --list-secret-keys --with-colons --fingerprint --fingerprint"
Unable to decrypt Data Encryption Key (DEK)

Following your comment in the issue #20 (link), I think the Github documentation isn't enough.

Could you provide a simple example to generate both keys, please?

being able to use kubesec with "non secret" files in YAML stream file

Hi,

It could be simpler to use to be able to run kubesec commands on multi "stream" yaml, even if doesn't contain only secrets.

For example, the file all.yaml:

apiVersion: v1
data:
  application.yaml: |2

    podcastserver:
      rootfolder: /podcasts
      concurrent-download: 40
      max-update-parallels: 256
      number-of-day-to-download: 30
      number-of-day-to-save-cover: 30

    spring:
      datasource.url: jdbc:h2:tcp://database:1521/podcast-server
      jpa.hibernate.ddl-auto: none
kind: ConfigMap
metadata:
  name: podcast-server-d989t5tgb4
  namespace: podcast-server
---
apiVersion: v1
data:
  api.youtube: ofajoeaijeoijeaoaeijeofiajaef
kind: Secret
metadata:
  name: podcast-server-b5cddm29mt
  namespace: podcast-server
type: Opaque

I would like to be able to run something like this :

$ cat all.yaml | kubesec -d - | kubectl apply -f -

But, right now, I have the following result:

image

kubesec ecrypt --key=... does not remove other keys

I use kubesec to encrypt kubernetes secrets. I update the secrets using
kubesec encrypt --in-place --key=... --key... --key....
I expect kubesec to encrypt to the specified keys only. Otherwise I would use the + option.
What really happens is that kubesec does not remove the keys that are not specified.

Integration with kustomize

After some (laborious) investigation, I conclude I can't use this very good plugin with a tool like kustomize because the standard generator doesn't allow to add some extra "comments" required by kubesec.

I can choose to not use the secret-generator from kustomize, but I lose auto rolling of deployment attached to a secret and the capacity to have immutable secrets in K8S.

Do you think an integration inside kustomize with an extension of the current secretGenerator (https://github.com/kubernetes-sigs/kustomize/tree/master/plugin/builtin/secretgenerator) or something else is possible? Would be great to have access to this 👍.

Thanks

/cc @Neonox31

kubesec not running on alpine linux

Problem

Trying to run kubesec in alpine linux results in:

bash-4.4# ./kubesec
bash: ./kubesec: No such file or directory

How to reproduce

$ docker build -t kubesec_bug https://gist.githubusercontent.com/neerfri/08b3f87a6ad9f5bdb9bf771616175407/raw/e8b8d578122f499bfbb63a8c49811d25b159e64b/Dockerfile
$ docker run --rm -it kubesec_bug /bin/sh -c '/root/kubesec'
$ docker run --rm -it kubesec_bug /bin/sh -c 'ldd /root/kubesec'

Notes

I believe the reason is that it's dynamically linked against libraries (glibc) that don't exist in alpine (by default).
You can see a similar example and suggestions for solutions in:
gliderlabs/docker-alpine#78
and https://github.com/blang/golang-alpine-docker

Support for custom IV

While I know that this is debatable, I have some scenarios where I would not like the file to physically change every time I encrypt it.

As example, if I run kubesec encrypt myfile, twice the output is not the same and I would like to have it to be the same.

The use case for this is something similar as git-crypt, where the encryption is deterministic (which is required so git can distinguish when a file has and hasn't changed).

Thanks

kubesec.io

There is another program with different purpose but same name. Very unfortunate for me who want to use both. :(
Check this. kubesec.io

Support working without application-default login

Hello. I have several gcloud instances for different purposes and I would like to avoid application-default login and use a local gcloud instance (which is supposed to be logged in) with kubesec. Is is possible somehow?

include license file in source code

Could you please include a copy of the MIT license along with the source code for this project? It would make the lives of people packaging and distributing it much easier.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.