cisco-sso / kdk Goto Github PK

https://github.com/mozilla/sops/releases

https://github.com/sstephenson/bats

https://developer.github.com/v3/#rate-limiting

Starting with Git Commit Hash: 38d2760

SSH keys are currently generated as root giving a permission denied on ./kdk ssh.

ssh kdk@localhost -A -p 2022 -i ~/.kdk/ssh/id_rsa -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null
Warning: Permanently added '[localhost]:2022' (ECDSA) to the list of known hosts.
Load key "/home/kdk/.kdk/ssh/id_rsa": Permission denied
kdk@localhost's password:

kdk@rb:~$ ls -aln ./.kdk/ssh/
total 16
drwxr-xr-x 2 1002 1002 4096 Jun 29 12:12 .
drwxr-xr-x 3 1002 1002 4096 Jun 29 12:12 ..
-rw------- 1    0    0 3247 Jun 29 12:12 id_rsa
-rw------- 1 1000 1000  743 Jun 29 12:12 id_rsa.pub

KDK doesn't work when using Python 3.7 i.e. via MacOS Homebew and Executor 20.0 due to asyncbeing a reserved keyword in Python 3.7 [1].

Traceback (most recent call last):
  File "./kdk", line 12, in <module>
    from executor import execute
  File "/Users/raobhati/Dev/kdk/.venv/lib/python3.7/site-packages/executor/__init__.py", line 199
    if command.async:
                   ^
SyntaxError: invalid syntax

Add HashiCorp Consul CLI https://www.consul.io/downloads.html

https://github.com/gruntwork-io/terragrunt/releases

Running kdk update without a valid config file results in the following runtime error:
panic: runtime error: invalid memory address or nil pointer dereference.

kdk update                                                                                                                                            
WARN[0000] Failed to load KDK config.                    configFileUsed=/Users/user/.kdk/kdk/config.yaml err="open /Users/user/.kdk/kdk/config.yaml: no such file or directory"
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x90 pc=0x15fd04a]

goroutine 1 [running]:
github.com/cisco-sso/kdk/pkg/kdk.needsUpdateConfig(...)
	/home/travis/gopath/src/github.com/cisco-sso/kdk/pkg/kdk/update.go:81
github.com/cisco-sso/kdk/pkg/kdk.Update(0x1df1880)
	/home/travis/gopath/src/github.com/cisco-sso/kdk/pkg/kdk/update.go:94 +0x95a
github.com/cisco-sso/kdk/cmd/kdk.glob..func11(0x1de3de0, 0x1e0e5b8, 0x0, 0x0)
	/home/travis/gopath/src/github.com/cisco-sso/kdk/cmd/kdk/update.go:27 +0x2d
github.com/spf13/cobra.(*Command).execute(0x1de3de0, 0x1e0e5b8, 0x0, 0x0, 0x1de3de0, 0x1e0e5b8)
	/home/travis/gopath/pkg/mod/github.com/spf13/[email protected]/command.go:766 +0x2aa
github.com/spf13/cobra.(*Command).ExecuteC(0x1de2880, 0xffffffffffffffff, 0x0, 0xc0004a9e08)
	/home/travis/gopath/pkg/mod/github.com/spf13/[email protected]/command.go:852 +0x2ea
github.com/spf13/cobra.(*Command).Execute(...)
	/home/travis/gopath/pkg/mod/github.com/spf13/[email protected]/command.go:800
github.com/cisco-sso/kdk/cmd/kdk.Execute()
	/home/travis/gopath/src/github.com/cisco-sso/kdk/cmd/kdk/kdk.go:49 +0x42
main.main()
	/home/travis/gopath/src/github.com/cisco-sso/kdk/main.go:20 +0x20

Had to manually remove the stopped container.

$ docker ps
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES
$ kdk up
FATA[0000] Failed to create KDK container                command=up error="Error response from daemon: Conflict. The container name \"/kdk\" is already in use by container \"60c11804a8d6e7353e2d5407be026375a93a0c9013b63baa51934781e1cce07c\". You have to remove (or rename) that container to be able to reuse that name."
$ docker ps -a
CONTAINER ID        IMAGE                        COMMAND                  CREATED             STATUS                      PORTS                    NAMES
60c11804a8d6        ciscosso/kdk:debian-latest   "/lib/systemd/systemd"   5 days ago          Exited (255) 23 hours ago   0.0.0.0:2022->2022/tcp   kdk
$ kdk destroy
INFO[0000] No KDK containers found. Nothing to destroy...  command=destroy
$ docker rm -f kdk
kdk
$ kdk up
INFO[0000] Successfully started KDK container            command=up

go get github.com/prometheus/alertmanager/cmd/amtool

https://github.com/prometheus/alertmanager#amtool

• Every time ssh, look for docker-for-desktop context in ~/.kube/config,
  parse it, create new file inside kdk at ~/.kube/docker-for-desktop,
  use golang tmp file lib.

Because of this:

#180

The octavia client is required to interact with openstack loadbalancers.

• Use scp to copy files to/from host and a kdk container.
• Should support naming one of multiple kdk containers.

Add HashiCorp Vault CLI https://www.vaultproject.io/downloads.html

kdk up fails on Windows because keybase requires Dokan 1.2.0 and the kdk expects 1.1.0. Downgrading Dokan to 1.1.0 causes keybase filesystem integration to be disabled.

The system cannot find the file C:\Program Files\Dokan\Dokan Library-1.1.0\sample\mirror\mirror.exe.
"Unrecognized parameter start. You must pass either start or stop"
INFO[0019] An exited KDK container exists

Unfortunately, Docker for Mac doesn't support IPV6, so we have ended up installing many kdk tools in a vagrant image.

https://github.com/mike-engel/jwt-cli

Example usage:

foo get token | jwt decode -

Token header
------------
{
  "typ": "JWT",
  "alg": "HS256"
}

Token claims
------------
{
  "aud": "api",
  "exp": 0,
  "https://example.org": {},
  "iat": 1582668632,
  "id": "b74647fd-cecd-5a7e-b672-5f00a1163eaf",
  "iss": "api",
  "role": "boss"
}

Is it a good idea to have a Dockerfile that can help to update chart repo (cisco-sso/charts)?

Mainly the Dockerfile will have awake, helm, kubectl, ansible installed.

Current KDK Dockerfile might be little big for that!

I have tried to build it, size is coming close to 300MB.

https://github.com/buildinspace/peru

https://pypi.org/project/click/

https://github.com/fluxcd/flux/releases

https://hub.docker.com/r/ciscosso/kdk/builds/bfeml4dy4tmcbacjv79jslu/ shows that the Docker container was build FROM debian:9.5 (see 9777ff4).

Once we add a new tag, we should also follow this change in

When doing a fresh install on MacOS 10.15.4, $HOME/.kdk/ does not exist.

The first run of kdk init is therefore generating warnings that are confusing to new users, specifically [...] "open /Users/user/.kdk/kdk/config.yaml: no such file or directory".

Additional information and/or clarification that this error is OK in such a case would improve the user experience.

Thanks,
Raoul

Full output of kdk init:

$ kdk init                                                                                                                                              
WARN[0000] Failed to load KDK config.                    configFileUsed=/Users/user/.kdk/kdk/config.yaml err="open /Users/user/.kdk/kdk/config.yaml: no such file or directory"
WARN[0000] Failed to add keybase mount:Failed to detect potential keybase filesystem mounts
Would you like to mount additional docker host directories into the KDK? [y/n] n
Would you like to enable SOCKS proxy? [y/n] n
INFO[0103] Set SOCKS port 8000
WARN[0103] KDK config does not exist
INFO[0103] Creating KDK config
WARN[0103] KDK ssh key pair not found.
INFO[0103] Generating ssh key pair...
INFO[0105] Successfully generated ssh key pair.
INFO[0105] KDK config written to /Users/user/.kdk/kdk/config.yaml. Modify this file to suit your needs.

We need to bring docker lib to newer version to support cached, ref: https://docs.docker.com/docker-for-mac/osxfs-caching/

Performance test was done with kubernetes repo under golang docker container and go list

for D in $(seq 1 5); do cd /perftest/ && time go list ./... > /dev/null; done 2>&1 | grep real | tail -3

Comparision:

1. linux does not have much difference:

• docker run -it --rm -v `pwd`/kubernetes:/perftest golang

real 0m4.099s
real 0m4.101s
real 0m3.981s

• docker run -it --rm -v `pwd`/kubernetes:/perftest:cached golang

real 0m3.807s
real 0m3.843s
real 0m4.031s

2. mac: w/ cached is about 3 times faster than w/o cached

• docker run -it --rm -v `pwd`/kubernetes:/perftest golang

real 2m8.093s
real 2m12.504s
real 2m34.471s

• docker run -it --rm -v `pwd`/kubernetes:/perftest:cached golang

real 0m40.561s
real 0m41.422s
real 0m41.870s

With macOS Catalina, you can no longer store files or data in the read-only system volume, nor can you write to the "root" directory ( / ) from the command line, such as with Terminal.

Read more at: https://support.apple.com/en-us/HT210650

How to configure KDK to change Keybase mount from default /keybase to custom path?

Crashes because awake currently hardcodes the path to python3.6 binary.

I destroy and recreate kdk containers quite often. It would be nice if we could allow the users to pass in the configuration as a string or from a file and avoid the interactive prompts alltogether.

something like this...

kdk init --config '{"Name": "kdk0", "DotfilesRepo": "https://github.com/rtluckie/work-dotfiles.git", "Shell": "/usr/local/bin/zsh", "KeybaseMount": true, "ExtraMounts": [{"Source": "/User/jdoe/projects", "Target": "/home/jdoe/projects"}, {"Source": "/User/jdoe/docs", "Target": "/home/jdoe/docs"}], "socks": {"enable": true, "port": 8000}}'

kdk init --config-from-file my-kdk-config.yaml

# my-kdk-config.yaml
Name: kdk0
DotfilesRepo: https://github.com/rtluckie/work-dotfiles.git
Dhell: "/usr/local/bin/zsh"
DeybaseMount: true
ExtraMounts:
- Source: "/User/jdoe/projects"
  Target: "/home/jdoe/projects"
- Source: "/User/jdoe/docs"
  Target: "/home/jdoe/docs"
Socks:
  enable: true
  port: 8000

https://github.com/google/yapf

https://tox.readthedocs.io/en/latest/

pip3 install yamllint

https://github.com/fullstorydev/grpcurl

https://github.com/go-task/task/releases

To demonstrate this tool works, here is very basic example Taskfile.yml:

version: "2"

tasks:

  default:
    cmds:
      - task: install

  install:
    desc: Install to Kubernetes.
    deps:
      - tiller
      - nginx-ingress
      - consul

  tiller:
    desc: Install Helm Tiller.
    cmds:
      - helm init --wait

  nginx-ingress:
    desc: Install Helm Chart for Nginx Ingress.
    cmds:
      - helm upgrade --install --recreate-pods
          --namespace nginx-ingress
          --values nginx-ingress/values.yaml
          nginx-ingress stable/nginx-ingress

  consul:
    desc: Install Helm Chart for Consul.
    cmds:
      - helm upgrade --install --recreate-pods
          --namespace consul
          --values consul/values.yaml
          consul stable/consul

I am facing the below issue with KDK while accessing behind VPN
Not able to access mounted K: and C: drive and getting the error="A firewall is blocking file sharing between windows and containers"
Error response from daemon: "Drive sharing seems blocked by a firewall"

However everything is working as expected while using Cisco network

Currently, the installation does not work with Macs with M1 chips.
I tried bypassing the installation.sh script and installed the amd64 version and was getting the following error with kdk ssh

kex_exchange_identification: Connection closed by remote host
Connection closed by ::1 port 59033
FATA[0011] Failed to ssh to KDK container.               error="exit status 255"

We need to get fully automated CI sorted out. The manual pushing of the docker containers is a serious PITA.

I would advocate moving from travis to circleci.

Run the following command to continue once this error has been fixed:
direnv allow
######################### DEPENDENCY ERROR #########################
######################### DEPENDENCY ERROR #########################
You must be registered on keybase.io and added to the team cisco_sso.honjo

Run the following command to continue once this error has been fixed:
direnv allow
######################### DEPENDENCY ERROR #########################
######################### DEPENDENCY ERROR #########################
Failed to find /keybase/team/cisco_sso.honjo/kubeconfig/honjo1-dev.csco.cloud

This error may be benign if the cluster hasn't been created yet

Run the following command to continue once this error has been fixed:
direnv allow

Hi Team,

I upgraded the KDK from 1.6.1 to 1.8.0 yesterday. Now the KDK container is not getting started. Getting the below error.

$ kdk ssh -d
time="2019-03-19T09:30:49+05:30" level=warning msg="Upgrade Available\n***************************************\nSome KDK components are out of date.\n Latest Version: 1.9.0\n Binary Version: 1.8.0\n Image Tag: 1.8.0\n Container Present at Config Version: false\n\nPlease upgrade the KDK with the commands:\n kdk update\n kdk destroy\n kdk ssh\n***************************************"
time="2019-03-19T09:30:49+05:30" level=info msg="Connecting to KDK container"
time="2019-03-19T09:30:49+05:30" level=info msg="KDK is not currently running. Starting..."
time="2019-03-19T09:30:49+05:30" level=debug msg="Not pulling already present KDK Image" tag=1.8.0
time="2019-03-19T09:30:49+05:30" level=info msg="Keybase mirror already started"
time="2019-03-19T09:30:50+05:30" level=fatal msg="Failed to create KDK container" error="Error response from daemon: invalid mount config for type "bind": bind source path does not exist: /host_mnt/c/Users/soupradh/.kdk/ssh/id_rsa.pub"

Not sure where it is taking the /host_mnt/.

I have updated to 1.9.0, but also getting the same error.

Thanks,
Soubhayga