Code Monkey home page Code Monkey logo

kubecrt's Introduction

kubecrt

Convert Helm charts to Kubernetes resources.

Description

kubecrt allows you to define your application's Kubernetes infrastructure based on a single configuration file.

The configuration file contains your application name, and namespace (both can also be set using CLI arguments), and you provide a list of charts that you want to install, optionally providing override values for those charts.

When running kubecrt, you provide it your project's configuration file, and in turn, it returns you the parsed Kubernetes resource files generated by the charts.

This allows you to use Helm Charts without actually using Helm, and instead using regular kubectl to deploy and manage your resources.

The configuration file you feed into kubecrt will be processed using the epp templating tool, allowing you to inject variables at runtime, based on your own conditional logic (production vs staging, etc...).

Installation

brew tap blendle/blendle
brew install kubecrt

Usage

See kubecrt --help

kubecrt - convert Helm charts to Kubernetes resources

Given a charts.yml file, compile all the charts with
the provided template variables and return the
resulting Kubernetes resource files to STDOUT, or
write them to a provided file.

Doing this, you can use Kubernetes charts, without
having to use Helm locally, or Tiller on the server.

Usage:
  kubecrt [options] CHARTS_CONFIG
  kubecrt -h | --help
  kubecrt --version
  kubecrt --example-config

Where CHARTS_CONFIG is the location of the YAML file
containing the Kubernetes Charts configuration.

Arguments:
  CHARTS_CONFIG                    Charts configuration file

Options:
  -h, --help                       Show this screen
  --version                        Show version
  -n NS, --namespace=NS            Set the .Release.Namespace chart variable,
                                   used by charts during compilation
  -a NAME, --name=NAME             Set the .Release.Name chart variable, used by
                                   charts during compilation
  -o PATH, --output=PATH           Write output to a file, instead of STDOUT
  -r NAME=URL, --repo=NAME=URL,... List of NAME=URL pairs of repositories to add
                                   to the index before compiling charts config
  -p DIR, --partials-dir=DIR       Path from which to load partial templates
                                   [default: config/deploy/partials]
  -j, --json                       Print resources formatted as JSON instead of
                                   YAML. Each resource is printed on a single
                                   line.
  --example-config                 Print an example charts.yaml, including
                                   extended documentation on the tunables

Charts Configuration File

See kubecrt --example-config

# apiVersion defines the version of the charts.yaml structure. Currently,
# only "v1" is supported.
apiVersion: v1

# name is the .Release.Name template value that charts can use in their
# templates, which can be overridden by the "--name" CLI flag. If omitted,
# "--name" is required.
name: my-bundled-apps

# namespace is the .Release.Namespace template value that charts can use in
# their templates. Note that since kubecrt does not communicate with
# Kubernetes in any way, it is up to you to also use this namespace when
# doing kubectl apply [...]. Can be overridden using "--namespace".  If omitted,
# "--namespace" is required.
namespace: apps

# charts is an array of charts you want to compile into Kubernetes resource
# files.
#
# A single chart might be used to deploy something simple, like a memcached pod,
# or something complex, like a full web app stack with HTTP servers, databases,
# caches, and so on.
charts:

# A Chart can either be in the format REPO/NAME, or a PATH to a local chart.
#
# If using REPO/NAME, kubecrt knows by-default where to locate the "stable"
# repository, all other repositories require the "repo" configuration (see
# below).
- stable/factorio:
    # values is a map of key/value pairs used when compiling the chart. This
    # uses the same format as in regular chart "values.yaml" files.
    #
    # see: https://git.io/v9Tyr
    values:
      resources:
        requests:
          memory: 1024Mi
          cpu: 750m
      factorioServer:
        # charts.yaml supports the same templating as chart templates do,
        # using the "sprig" library.
        #
        # see: https://masterminds.github.io/sprig/
        name: >
          {{ env "MY_SERVER_NAME" | default "hello world!" }}

- stable/minecraft:
    # version is a semantic version constraint.
    #
    # see: https://github.com/Masterminds/semver#basic-comparisons
    version: ~> 0.1.0
    values:
      minecraftServer:
        difficulty: hard

- opsgoodness/prometheus-operator:
    # repo is the location of a repositry, if other than "stable". This is
    # the URL you would normally add using "helm repo add NAME URL".
    repo: http://charts.opsgoodness.com
    values:
      sendAnalytics: false

# For the above charts, see here for the default configurations:
#
#   * stable/factorio: https://git.io/v9Tyr
#   * stable/minecraft: https://git.io/v9Tya
#   * opsgoodness/prometheus-operator: https://git.io/v9SAY

Partial Templates

You can optionally split your charts.yml file into multiple chunks, by using partial templates. This works almost the same way as Helm's support for these in charts. See the Helm documentation for more details.

To use these partials, you have to set the --partials-dir flag when calling kubecrt, pass it the path to your partials directory, and then use those partials in your charts.yml.

Example:

charts.yml:

apiVersion: v1
name: my-bundled-apps
namespace: apps
charts:
- stable/factorio:
    values:
      resources:
{{ include "factorio/resources" . | indent 8 }}

partials/factorio/resources.yml

{{- define "factorio/resources" -}}
requests:
  memory: 1024Mi
  cpu: 750m
{{- end -}}

You can then run this as follows:

kubecrt --partials-dir ./partials charts.yml

And the result is a fully-parsed charts printed to stdout.

Some notes:

  • you can use subfolders to organise your partials
  • each named define has to be uniquely named, or risk being overwritten
  • you can define multiple define blocks in a single file
  • the files don't need to be yaml files, you can use any content you need

Releasing new version

make (major|minor|patch)
git push --tags
make dist
open _dist/

Next, go to the GitHub releases page, and edit the tag you just push:

  • Release title: vx.x.x
  • Describe this release: short description of important changes
  • Attach binaries: drop the files created in _dist here

Click "Update release".

Don't forget to update the Homebrew formula, located at blendle/homebrew-blendle.

kubecrt's People

Contributors

jeanmertz avatar oscaredel avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

kubecrt's Issues

fix panic for unsatisfiable version constraint

If version is set to an unsatisfiable constraint (for example, a version is specified that does not exist), a panic is currently raised:

panic: runtime error: index out of range

goroutine 1 [running]:
github.com/blendle/kubecrt/helm.GetAcceptableVersion(0xc420dd2130, 0xb, 0xc420dd2148, 0x8, 0x0, 0x11, 0xc420bc1460, 0x19)
        /private/tmp/kubecrt-20170513-15659-nlayje/src/github.com/blendle/kubecrt/helm/version.go:35 +0x114
github.com/blendle/kubecrt/chart.locateChartPath(0xc420dd2130, 0xb, 0xc420dd2148, 0x8, 0x1075985, 0x4, 0xc4206e3bd0, 0x101699f)
        /private/tmp/kubecrt-20170513-15659-nlayje/src/github.com/blendle/kubecrt/chart/parser.go:190 +0x589
github.com/blendle/kubecrt/chart.(*Chart).compile(0xc420bba500, 0x7fff5fbffa4d, 0x11, 0x7fff5fbffa2c, 0x19, 0xc420bba680, 0x3a, 0x1, 0x1, 0x1, ...)
        /private/tmp/kubecrt-20170513-15659-nlayje/src/github.com/blendle/kubecrt/chart/parser.go:72 +0x61
github.com/blendle/kubecrt/chart.(*Chart).ParseChart(0xc420bba500, 0x7fff5fbffa4d, 0x11, 0x7fff5fbffa2c, 0x19, 0x0, 0x0, 0x0, 0x0, 0x0)
        /private/tmp/kubecrt-20170513-15659-nlayje/src/github.com/blendle/kubecrt/chart/parser.go:61 +0x253
github.com/blendle/kubecrt/chartsconfig.(*ChartsConfiguration).ParseCharts(0xc420db34a0, 0x0, 0x0, 0xc420db34a0, 0x0, 0x0)
        /private/tmp/kubecrt-20170513-15659-nlayje/src/github.com/blendle/kubecrt/chartsconfig/parser.go:49 +0xa6
main.main()
        /private/tmp/kubecrt-20170513-15659-nlayje/src/github.com/blendle/kubecrt/main.go:72 +0x92d

We should instead inform the user that an acceptable version could not be found in the currently supplied repositories.

support chart repositories behind SSL certificate authentication

Our current API looks like this:

apiVersion: v1
charts:
- opsgoodness/prometheus-operator:
    repo: http://charts.opsgoodness.com

We should also support SSL-based authentication, potentially with an API like this:

apiVersion: v1
charts:
- opsgoodness/prometheus-operator:
    repo:
      url: https://charts.opsgoodness.com
      caFile: /path/to/ca
      certFile: /path/to/cert
      keyFile: /path/to/key

However, given that we support templating within charts.yml, we could also consider to not point to files, but instead embed the data as base64-encoded strings, as this makes it easier to inject the values in an automated/CI-environment using environment variables:

apiVersion: v1
charts:
- opsgoodness/prometheus-operator:
    repo:
      url: https://charts.opsgoodness.com
      ca: {{ env "MY_CA_CONTENT" | b64enc }}
      cert: {{ env "MY_CERT_CONTENT" | b64enc }}
      key: {{ env "MY_KEY_CONTENT" | b64enc }}

Or support both formats, if that makes sense.

Wrong line number returned in error description

See #21.

I suspect this is happening here:

func errorLine(err error) (int, string) {
var i int
var p []string
str := err.Error()
println(str)
if strings.HasPrefix(str, "yaml: ") {
p = strings.SplitN(str, ":", 3)
i, _ = strconv.Atoi(strings.Replace(p[1], " line ", "", -1))
str = strings.TrimSpace(p[2])
}
if strings.HasPrefix(str, "template: test:") {
p = strings.SplitN(str, ":", 4)
i, _ = strconv.Atoi(p[2])
str = strings.TrimSpace(p[3])
}
return i, "Templating error: " + str
}

Support partial templates

Helm supports the use of golang partial template, it would be nice if kubecrt would support this as well.
This would enable us to spread large config files across multiple files so is easier to maintain them.

Add ability to share common / global env and secrets

Some projects (especially the old ones), have a lot of envs & secrets configured. If you have a couple of deployments, you need to change them for all those deployments. Having a way of centralising those values would be very helpful.

Some ideas:

indentation error when converting defectdojo

I tried to download and convert local helm chart downloaded from product page here:
https://github.com/DefectDojo/django-DefectDojo/blob/master/KUBERNETES.md

When converting the helm chart it it creates an error in the section below, where "requests" on the same level as "resources" but it must be indented as "requests" is subkey of resource. Interestingly on other blocks which have exactly the same dfinition it works perfectly Here's the wrong code block:

apiVersion: batch/v1
kind: Job
metadata:
  name: defectdojo-initializer
  labels:
    defectdojo.org/component: initializer
    app.kubernetes.io/name: defectdojo
    app.kubernetes.io/instance: defectdojo
    app.kubernetes.io/managed-by: Tiller
    helm.sh/chart: defectdojo-0.1.0
spec:
  ttlSecondsAfterFinished: 
  template:
    metadata:
      labels:
        defectdojo.org/component: initializer
        app.kubernetes.io/name: defectdojo
        app.kubernetes.io/instance: defectdojo
    spec:
      containers:
        - name: initializer
          image: "defectdojo/defectdojo-django:latest"
          imagePullPolicy: Always
          command: ['/entrypoint-initializer.sh']
          envFrom:
            - configMapRef:
                name: defectdojo
            - secretRef:
                name: defectdojo
          env:
            - name: DD_DATABASE_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: defectdojo-mysql
                  key: mysql-password
          resources:
          requests:
            cpu: 100m
            memory: 128Mi
          
      restartPolicy: Never
  backoffLimit: 1

add namespace to generated kuberenetes output

Hi,

I think to really make use of kubecrt in a proper way it should set the namespace by creating it first through:

 apiVersion: v1
    kind: Namespace
    metadata:
      name: weave
      annotations:
        cloud.weave.works/version: v1.0.0-259-g2850b85

and then reference it in the various other yamls by the metadata/namespace key. For automated deployment this is necessary because the yamls are automatically applied and not manually through kubectl with namespace beeing defined there.

add top-level "repositories" object

Our current API looks like this:

apiVersion: v1
charts:
- blendle/web:
    repo: https://blendle-charts.storage.googleapis.com

While this works, it becomes tedious if you want to add multiple charts from the same (non-stable/default) repository:

apiVersion: v1
charts:
- blendle/web:
    repo: https://blendle-charts.storage.googleapis.com
- blendle/cron:
    repo: https://blendle-charts.storage.googleapis.com

The current solution to this is to launch kubecrt using the --repo flag:

kubecrt --repo="blendle=https://blendle-charts.storage.googleapis.com" charts.yml

After which you no longer need to define the repo in the charts/yml:

apiVersion: v1
charts:
- blendle/web: {}
- blendle/cron: {}

This works, but if we want charts.yml to be a self-sustainable config, that does not depend on CLI flags, we could introduce a new repositories object:

apiVersion: v1

repositories:
  blendle: https://blendle-charts.storage.googleapis.com

charts:
- blendle/web: {}
- blendle/cron: {}

Can't tap from homebrew

When I do brew tap blendle/blendle on my M1 Mac I got:

Error: Invalid formula: /opt/homebrew/Library/Taps/blendle/homebrew-blendle/Formula/kubecrt.rb
kubecrt: Calling cellar in a bottle block is disabled! Use brew style --fix on the formula to update the style or use sha256 with a cellar: argument instead.
Please report this issue to the blendle/blendle tap (not Homebrew/brew or Homebrew/core), or even better, submit a PR to fix it:
/opt/homebrew/Library/Taps/blendle/homebrew-blendle/Formula/kubecrt.rb:11

Error: Invalid formula: /opt/homebrew/Library/Taps/blendle/homebrew-blendle/Formula/epp.rb
epp: Calling cellar in a bottle block is disabled! Use brew style --fix on the formula to update the style or use sha256 with a cellar: argument instead.
Please report this issue to the blendle/blendle tap (not Homebrew/brew or Homebrew/core), or even better, submit a PR to fix it:
/opt/homebrew/Library/Taps/blendle/homebrew-blendle/Formula/epp.rb:11

What can I do to resolve this?

Templating error: did not find expected key - improve error messages

-> % kubecrt chart.yml
yaml: line 36: did not find expected key
charts config parsing error: 

 1: 
 2: # apiVersion defines the version of the charts.yaml structure. Currently,
 3: # only "v1" is supported.
 4: apiVersion: v1
 5: 
 6: # name is the .Release.Name template value that charts can use in their
 7: # templates, which can be overridden by the "--name" CLI flag. If omitted,
 8: # "--name" is required.
 9: name: loki
10: 
11: # namespace is the .Release.Namespace template value that charts can use in
12: # their templates. Note that since kubecrt does not communicate with
13: # Kubernetes in any way, it is up to you to also use this namespace when
14: # doing kubectl apply [...]. Can be overridden using "--namespace".  If omitted,
15: # "--namespace" is required.
16: namespace: prometheus
17: 
18: # charts is an array of charts you want to compile into Kubernetes resource
19: # files.
20: #
21: # A single chart might be used to deploy something simple, like a memcached pod,
22: # or something complex, like a full web app stack with HTTP servers, databases,
23: # caches, and so on.
24: charts:
25: 
26: # A Chart can either be in the format REPO/NAME, or a PATH to a local chart.
27: #
28: # If using REPO/NAME, kubecrt knows by-default where to locate the "stable"
29: # repository, all other repositories require the "repo" configuration (see
30: # below).
31: - "/home/kris/PycharmProjects/loki/production/helm/loki-stack/Chart.yaml"
32: 
33:     # values is a map of key/value pairs used when compiling the chart. This
34:     # uses the same format as in regular chart "values.yaml" files.
35:     #
36:     # see: https://git.io/v9Tyr
37:     values:
38:       loki:
39:         enabled: true
40:       promtail:
41:         enabled: true
42:       grafana:
43:         enabled: false
44:         sidecar:
45:           datasources:
46:             enabled: true
47:       image:
48:         tag: 6.3.0-beta2
49: 
50:     prometheus:
51:       enabled: false
52:       server:
53:         fullnameOverride: prometheus-server
54: 
55:         requests:
56:           memory: 1024Mi
57:           cpu: 750m
58:         # charts.yaml supports the same templating as chart templates do,
59:         # using the "sprig" library.
60:         #
61:         # see: https://masterminds.github.io/sprig/
62: 
63: 
64: # For the above charts, see here for the default configurations:
65: #
66: #   * stable/factorio: https://git.io/v9Tyr
67: #   * stable/minecraft: https://git.io/v9Tya
68: #   * opsgoodness/prometheus-operator: https://git.io/v9SAY


33:     # values is a map of key/value pairs used when compiling the chart. This
34:     # uses the same format as in regular chart "values.yaml" files.
35:     #
36:     # see: https://git.io/v9Tyr
37:     values:
38:       loki:
39:         enabled: true

Templating error: did not find expected key

This is my first time using this tool. I'd like to know what went wrong, which variable was unset or what key was expected and where. The error message highlights line 36 which is a comment so I guess it's about line 37?

allow list of naked charts

Our current API looks like this:

apiVersion: v1
charts:
- blendle/web:
    repo: https://blendle-charts.storage.googleapis.com

If you simply want to compile a stable chart, without any custom configuration, you might end up with something like this:

apiVersion: v1
charts:
- stable/redis: {}
- stable/datadog: {}

Let's fix this, and also support a list of naked charts:

apiVersion: v1
charts:
- stable/redis
- stable/datadog

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.