amitkumardas / decide Goto Github PK

View Code? Open in Web Editor NEW

0.0 0.0 1.0 4.91 MB

License: Apache License 2.0

Makefile 1.52% Go 98.48%

decide's People

Watchers

decide's Issues

Explore Binary, GOMAXPROCS, Signal Handler, Log Setup etc -- startup logic

https://github.com/jetstack/navigator/pull/74/files

Evolution of CAS Template config

At CAS Template

spec:
  defaultConfig:
  - name: VolumeMonitor
    enabled: "true"
  - name: ControllerImage
    value: openebs/jiva:0.5.0
  - name: ReplicaImage
    value: openebs/jiva:0.5.0
  - name: ReplicaCount
    value: "1"
  - name: StoragePool
    value: jiva-pool-path-default-0.7.0
  - name: TaintTolerations
    value: |-
      t1:
        key: node.openebs.io/disktype
        operator: Equal
        value: ssd
        effect: NoSchedule
      t2:
        key: node.openebs.io/disktype
        operator: Equal
        value: ssd
        effect: NoExecute
  - name: EvictionTolerations
    value: |-
      t1:
        effect: NoExecute
        key: node.alpha.kubernetes.io/notReady
        operator: Exists
      t2:
        effect: NoExecute
        key: node.alpha.kubernetes.io/unreachable
        operator: Exists
  - name: NodeAffinityRequiredSchedIgnoredExec
    value: |-
      t1:
        key: beta.kubernetes.io/os
        operator: In
        values:
        - linux
  - name: NodeAffinityPreferredSchedIgnoredExec
    value: |-
      t1:
        key: some-node-label-key
        operator: In
        values:
        - some-node-label-value

Usage at RunTask

    {{- $isTaintTolerations := .Config.TaintTolerations.value | default "false" -}}
    {{- $taintTolerationsVal := fromYaml .Config.TaintTolerations.value -}}
    {{- $isEvictionTolerations := .Config.EvictionTolerations.value | default "false" -}}
    {{- $evictionTolerationsVal := fromYaml .Config.EvictionTolerations.value -}}
    spec:
      template:
        spec:
          tolerations:
          {{- if ne $isTaintTolerations "false" }}
          {{- range $k, $v := $taintTolerationsVal }}
          - 
          {{- range $kk, $vv := $v }}
            {{ $kk }}: {{ $vv }}
          {{- end }}
          {{- end }}
          {{- end }}
          {{- if ne $isEvictionTolerations "false" }}
          {{- range $k, $v := $evictionTolerationsVal }}
          - 
          {{- range $kk, $vv := $v }}
            {{ $kk }}: {{ $vv }}
          {{- end }}
          {{- end }}
          {{- end }}

Explore GOmega functions as templated functions

Explore helm vis-a-vis go template/text library

Ref - https://github.com/helm/charts/blob/master/stable/percona-xtradb-cluster/templates/statefulset.yaml

Explore logging libraries

Unit Testing as a REPL strategy to verify your hypothesis

translate any json and/or yaml as k8s resource

ref: https://github.com/kubernetes-sigs/kustomize/blob/master/pkg/crds/crds.go

import (
  "encoding/json"

  "github.com/ghodss/yaml"
  "k8s.io/apimachinery/pkg/runtime/schema"
  "k8s.io/kube-openapi/pkg/common"
)

package crds

// Annotation is to mark a field as annotations.
// "x-kubernetes-annotation": ""
const Annotation = "x-kubernetes-annotation"

// LabelSelector is to mark a field as LabelSelector
// "x-kubernetes-label-selector": ""
const LabelSelector = "x-kubernetes-label-selector"

// Identity is to mark a field as Identity
// "x-kubernetes-identity": ""
const Identity = "x-kubernetes-identity"

// Version marks the type version of an object ref field
// "x-kubernetes-object-ref-api-version": <apiVersion name>
const Version = "x-kubernetes-object-ref-api-version"

// Kind marks the type name of an object ref field
// "x-kubernetes-object-ref-kind": <kind name>
const Kind = "x-kubernetes-object-ref-kind"

// NameKey marks the field key that refers to an object of an object ref field
// "x-kubernetes-object-ref-name-key": "name"
// default is "name"
const NameKey = "x-kubernetes-object-ref-name-key"

	var types map[string]common.OpenAPIDefinition
	if content[0] == '{' {
		err = json.Unmarshal(content, &types)
		if err != nil {
			return nil, err
		}
	} else {
		err = yaml.Unmarshal(content, &types)
		if err != nil {
			return nil, err
		}
	}

// getCRDs get all CRD types
func getCRDs(types map[string]common.OpenAPIDefinition) map[string]schema.GroupVersionKind 
{
	crds := map[string]schema.GroupVersionKind{}

	for typename, t := range types {
		properties := t.Schema.SchemaProps.Properties
		_, foundKind := properties["kind"]
		_, foundAPIVersion := properties["apiVersion"]
		_, foundMetadata := properties["metadata"]
		if foundKind && foundAPIVersion && foundMetadata {
                  // TODO: Get Group and Version for CRD from the openAPI definition once
		  // "x-kubernetes-group-version-kind" is available in CRD
		  kind := strings.Split(typename, ".")[len(strings.Split(typename, "."))-1]
		  crds[typename] = schema.GroupVersionKind{Kind: kind}
		}
	}
	return crds
}

types map[string]common.OpenAPIDefinition

for propname, property := range types[atype].Schema.SchemaProps.Properties {
	_, annotate := property.Extensions.GetString(Annotation)
	if annotate {...}
	_, label := property.Extensions.GetString(LabelSelector)
	if label {...}
	_, identity := property.Extensions.GetString(Identity)
	if identity {...}
	version, ok := property.Extensions.GetString(Version)
	if ok {
		kind, ok := property.Extensions.GetString(Kind)
		if ok {
			nameKey, ok := property.Extensions.GetString(NameKey)
			if !ok {...}
		}
	}

	if property.Ref.GetURL() != nil {...}
}

Explore folder structure suitable for Ginkgo

Explore declarative way to expose kubectl & related operations

Expose kubectl operations as go template functions

The use of CAS Templates in openebs control plane enables defining operations via yaml i.e. declarative approach. The main motive is to make faster releases without having to go through code compilation & related dependencies. The other equally important motive is to enable openebs easier for community & user contribution. In other words make openebs contributions decentralised.

In light of above goods, it becomes imperative to make this declarative approach simpler to understand, use, execute, rollback & debug in case of failures. Below is a high level approach (read v1beta1 version) to ensure true effectiveness of CAS Templates.

Notes:

kubectl template functions
ability to choose one or more template libraries
remove sprig functions selectively if required
design your output
design your debug
design your rollback
make use of natural range vs use of repeatWith
avoid use of complex template functions
ability to run bulk of tasks in a RunTask
ability to run series of RunTasks
ability to make the cas template engine simple

A Sample RunTask:

metadata:
  name: DoItTask
spec:
  id: MyCoolRT
  # list of yamls
  yamls:
  - id: svcYaml
    content: |
      apiVersion: v1
      kind: Service
      metadata:
        labels:
          openebs.io/controller-service: jiva-controller-svc
          openebs.io/storage-engine-type: jiva
          openebs.io/pv: {{ .Volume.owner }}
        name: {{ .Volume.owner }}-ctrl-svc
      spec:
        ports:
        - name: iscsi
          port: 3260
          protocol: TCP
          targetPort: 3260
        - name: api
          port: 9501
          protocol: TCP
          targetPort: 9501
        selector:
          openebs.io/controller: jiva-controller
          openebs.io/pv: {{ .Volume.owner }}
  # list of executions
  executions: 
  - >
  {{/*
  Create a service based on provided yaml and store its ip and name
  */}}
  {{-  kubectl "MyPutSvc" |
      yaml .MyCoolRT.svcYaml |
      put "svc" |
      jsonpath "{range .items[*]}ip={@.spec.clusterIP},name={@.metadata.name};{end}" |
      itemSplit ";" |
      pairSplit ", =" |
      pick "ip" "MyCoolRT.MyPutSvc.ip" |
      pick "name" "MyCoolRT.MyPutSvc.name" |
      build -}}
   - >
  {{/*
  Get service based on name and store its cluster ip
  */}}
  {{-  kubectl "MyGetSvc" |
     get "svc" |
     name .Volume.owner |
     ns .Volume.runNamespace |
     jsonpath "{.spec.clusterIP}" |
     labelSelector .Volume.options |
     set "MyCoolRT.MyGetSvc.clusterIP"  |
     build -}}
  - >
  {{/*
  List services based on selector and store each service's ip and name
  */}}
  {{-  kubectl "MyListSvc" |
      list "svc" |
      labelSelector .Volume.options |
      jsonpath "{range .items[*]}ip={@.spec.clusterIP},name={@.metadata.name};{end}" |
      itemSplit ";" |
      pairSplit ", =" |
      pick "ip" "MyCoolRT.MyListSvc.ip" |
      pick "name" "MyCoolRT.MyListSvc.name" |
      build -}}
  rollbacks:
  - blah blah
  - blah blah

Kubernetes operators, controllers, watchers -- concepts

kubernetes filtering vs. watch

A controller that has to frequently filter down a list of thousands of things should probably use a watch instead.

Explore json query libraries

functional - https://github.com/thedevsaddam/gojsonq/
performance - https://github.com/json-iterator/go/

References:

Explore transformers of kustomize

https://github.com/kubernetes-sigs/kustomize/tree/master/pkg/transformers

Explore Reconcile Loop as templated functions

k8s.io/metacontroller/dynamic/clientset
k8s.io/metacontroller/dynamic/apply

Explore share memory by communicating

optional method interface via type casting and/or closures

 type eventHandle interface {
     eventHookTypeA
     eventHookTypeB
     // DON'T add TypeC interface.
     someOtherMethod() results
 }
 type eventHookTypeA interface {
     // method definitions
     HandleEventA() results
 }
 type eventHookTypeB interface {
     // method definitions
     HandleEventB() results
 }
 type eventHookTypeC interface {
     HandleEventC() results
 }

 func handleEvents(eventType string, handle eventHandle) results {
      if eventType == "eventA" {
         return handle.HandleEventA()
      } else if eventType == "eventB" {
         return handle.HandleEventB()
      } else if eventType == "eventC" {
         if c,ok := handle.(eventHookTypeC); ok {
             return c.HandleEventC()
         } else {
             log.Println("somewhat bad happen")
         }
      }
      return nil
 }

Explore - From functional book into template function

https://mostly-adequate.gitbooks.io/mostly-adequate-guide/

Explore Jsoniter as templated function

Design Thinking - Part 1

Types of program structures e.g. `go struct`:

API
Interface
Service

API files will have:

Middleware Functions i.e. decorator functions
Methods to enable Fluent APIs i.e. method chaining
error aggregation
Interface
- Single method interface
Service
- Composes of one or more interfaces
- Implement some interfaces

Directory structure - APIs

pkg/apis/v1/doc.go
- a bean & related interfaces will go in this file
similarly other files can be:
- doc_values.go,
- run_task_specs.go,
- label.go,
- field.go,
- select.go,
- query.go,
- cas_template_specs.go

Directory structure - Services

pkg/engine/v1/cas_template_engine.go
- a service that provide cas template engine
pkg/task/v1/run_task.go
- a service that can run one or more tasks
pkg/template/v1/task_templates.go
- a service that provides a bunch of task related go-template functions

Some Interface examples - Think Out of Box

DocGetter, ValuesGetter, SpecsGetter, LabelGetter, FieldGetter, SelectGetter,
QueryGetter, Marshaller, PodCreator, PodGetter, PodLister, PodDeleter,
JsonQuery, ValuePicker, etc

Poke The Box - Plans

Plan 16 Jul 2018

kubectl as template functions
kuberneter resource / operator installer
jsoniter as template functions
fio as template functions
easy to do integration testing
easy to visualize results of integration testing

Explore declarative code for kubernetes pod chaos testing

References:

https://github.com/jnewland/kubernetes-pod-chaos-monkey

amitkumardas / decide Goto Github PK

decide's People

Watchers

decide's Issues

At CAS Template

Usage at RunTask

Expose kubectl operations as go template functions

Notes:

A Sample RunTask:

kubernetes filtering vs. watch

References:

Types of program structures e.g. go struct:

API files will have:

Directory structure - APIs

Directory structure - Services

Some Interface examples - Think Out of Box

Plan 16 Jul 2018

References:

Recommend Projects

Recommend Topics

Recommend Org

Types of program structures e.g. `go struct`: