At the moment both make run and kubesim mount ~/.ssh to store the ssh keys used to connect to the cluster. There is no obvious need to store the keys there since they could be in ~/.kubesim instead. We are exposing unrelated user ssh keys to the container, something we should avoid as a matter of best practice.

The attack container uses a white colour for its welcome text. On light terminal themes this can blend into the background making it very hard or impossible to read.

A blue colour might work here since I don't think blue is used anywhere in the simulator CLIs.

You can find correct starting points in challenge.txt

Currently the user cannot supply a custom repo for the attack container. This means changes to the attack container can only be tested if a member of controlplane pushed a tag on behalf of them. This is a barrier to contributing.

Having a brief description in the scenarios list will ease the usability for the end user by providing useful information on the scenario itself so they can choose accordingly.

For its implementation:

• Modify the file: simulation-scripts/scenarios.yaml and add the description field.
• Modify function newScenarioListCommand in /cmd/scenario.go and add the description field into the string.

Use the same bash configurations for all instances and containers:

• Prompt command / PS1
• inputrc
• aliases
• bash_completion

Take from the attack/launch containers which @sublimino modified already to make more friendly

Currently the terraform directory is mounted read/write this has the potential for conflicts to occur if files are modified inside the container. We should mount it readonly.

The golang code writes out the bastion.tfvars file in that directory so we will need to change the location that gets written to and adjust the path to it given to terraform

This ticket has been split out from #102

We currently use the AWS golang SDK to create the terraform state bucket. It is only used for this purpose, all other interaction with AWS is done with terraform. The AWS instroduces a deep messy dependency graph that massively slows sown the build and increases the complexity of simulator. We should investigate using the AWS HTTP API directly to reduce the build times and remove the AWS SDK dependency.

1. Create infrastructure from IP 1
2. Connect from IP 2
3. Run simulator scenario launch node-shock-tactics
4. SSH timeout is long, then errors as the TCP connection times out at the security group

Fix:

1. Run simulator infra create a second time when connected to IP 2 -- this creates the appropriate security group rules and then scenario deployment works

1.5733158573249292e+09  INFO    simulator/launch.go:61  Setting up the "Node Shock Tactics" scenario on the cluster
ssh: connect to host 3.9.118.211 port 22: Connection timed out
[2019-11-09T16:13:08+0000] ./perturb.sh: No hash found
[2019-11-09T16:13:08+0000] ./perturb.sh: Found scenario 
[2019-11-09T16:13:08+0000] ./perturb.sh: Running scenario/node-shock-tactics/ against 172.31.2.37
Connection timed out during banner exchange
[2019-11-09T16:13:11+0000] ./perturb.sh: Cannot connect to 172.31.2.37
1.5733159911121545e+09  ERROR   cmd/scenario.go:54      Error launching scenario        {"error": "Error running perturb with simulator.PerturbOptions{Bastion:net.IP{0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 
0x0, 0x0, 0xff, 0xff, 0x3, 0x9, 0x76, 0xd3}, Master:net.IP{0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0xff, 0xff, 0xac, 0x1f, 0x2, 0x25}, Slaves:[]net.IP{net.IP{0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0
, 0x0, 0x0, 0xff, 0xff, 0xac, 0x1f, 0x2, 0x33}, net.IP{0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0xff, 0xff, 0xac, 0x1f, 0x2, 0x2b}}, ScenarioName:\"node-shock-tactics\"}: Error waiting for child pr
ocess ./perturb.sh: exit status 3", "errorVerbose": "exit status 3\nError waiting for child process ./perturb.sh\ngithub.com/controlplaneio/simulator-standalone/pkg/util.Run\n\t/go/src/github.com/controlpla
neio/simulator-standalone/pkg/util/run.go:58\ngithub.com/controlplaneio/simulator-standalone/pkg/simulator.Perturb\n\t/go/src/github.com/controlplaneio/simulator-standalone/pkg/simulator/perturb.go:76\ngith
ub.com/controlplaneio/simulator-standalone/pkg/simulator.Launch\n\t/go/src/github.com/controlplaneio/simulator-standalone/pkg/simulator/launch.go:62\ngithub.com/controlplaneio/simulator-standalone/cmd.newSc
enarioLaunchCommand.func1\n\t/go/src/github.com/controlplaneio/simulator-standalone/cmd/scenario.go:49\ngithub.com/spf13/cobra.(*Command).execute\n\t/home/build/go/pkg/mod/github.com/spf13/[email protected]/comm
and.go:826\ngithub.com/spf13/cobra.(*Command).ExecuteC\n\t/home/build/go/pkg/mod/github.com/spf13/[email protected]/command.go:914\ngithub.com/spf13/cobra.(*Command).Execute\n\t/home/build/go/pkg/mod/github.com/
spf13/[email protected]/command.go:864\ngithub.com/controlplaneio/simulator-standalone/cmd.Execute\n\t/go/src/github.com/controlplaneio/simulator-standalone/cmd/root.go:109\nmain.main\n\t/go/src/github.com/contr
olplaneio/simulator-standalone/main.go:13\nruntime.main\n\t/usr/lib/go-1.11/src/runtime/proc.go:201\nruntime.goexit\n\t/usr/lib/go-1.11/src/runtime/asm_amd64.s:1333\nError running perturb with simulator.Per
turbOptions{Bastion:net.IP{0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0xff, 0xff, 0x3, 0x9, 0x76, 0xd3}, Master:net.IP{0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0xff, 0xff, 0xac, 0x1f, 0x2, 0
x25}, Slaves:[]net.IP{net.IP{0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0xff, 0xff, 0xac, 0x1f, 0x2, 0x33}, net.IP{0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0xff, 0xff, 0xac, 0x1f, 0x2, 0x2b}
}, ScenarioName:\"node-shock-tactics\"}\ngithub.com/controlplaneio/simulator-standalone/pkg/simulator.Launch\n\t/go/src/github.com/controlplaneio/simulator-standalone/pkg/simulator/launch.go:64\ngithub.com/
controlplaneio/simulator-standalone/cmd.newScenarioLaunchCommand.func1\n\t/go/src/github.com/controlplaneio/simulator-standalone/cmd/scenario.go:49\ngithub.com/spf13/cobra.(*Command).execute\n\t/home/build/
go/pkg/mod/github.com/spf13/[email protected]/command.go:826\ngithub.com/spf13/cobra.(*Command).ExecuteC\n\t/home/build/go/pkg/mod/github.com/spf13/[email protected]/command.go:914\ngithub.com/spf13/cobra.(*Command).
Execute\n\t/home/build/go/pkg/mod/github.com/spf13/[email protected]/command.go:864\ngithub.com/controlplaneio/simulator-standalone/cmd.Execute\n\t/go/src/github.com/controlplaneio/simulator-standalone/cmd/root.

go:109\nmain.main\n\t/go/src/github.com/controlplaneio/simulator-standalone/main.go:13\nruntime.main\n\t/usr/lib/go-1.11/src/runtime/proc.go:201\nruntime.goexit\n\t/usr/lib/go-1.11/src/runtime/asm_amd64.s:1
333"}```
1.5733158573249292e+09  INFO    simulator/launch.go:61  Setting up the "Node Shock Tactics" scenario on the cluster
ssh: connect to host 3.9.118.211 port 22: Connection timed out
[2019-11-09T16:13:08+0000] ./perturb.sh: No hash found
[2019-11-09T16:13:08+0000] ./perturb.sh: Found scenario 
[2019-11-09T16:13:08+0000] ./perturb.sh: Running scenario/node-shock-tactics/ against 172.31.2.37
Connection timed out during banner exchange
[2019-11-09T16:13:11+0000] ./perturb.sh: Cannot connect to 172.31.2.37
1.5733159911121545e+09  ERROR   cmd/scenario.go:54      Error launching scenario        {"error": "Error running perturb with simulator.PerturbOptions{Bastion:net.IP{0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 
0x0, 0x0, 0xff, 0xff, 0x3, 0x9, 0x76, 0xd3}, Master:net.IP{0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0xff, 0xff, 0xac, 0x1f, 0x2, 0x25}, Slaves:[]net.IP{net.IP{0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0
, 0x0, 0x0, 0xff, 0xff, 0xac, 0x1f, 0x2, 0x33}, net.IP{0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0xff, 0xff, 0xac, 0x1f, 0x2, 0x2b}}, ScenarioName:\"node-shock-tactics\"}: Error waiting for child pr
ocess ./perturb.sh: exit status 3", "errorVerbose": "exit status 3\nError waiting for child process ./perturb.sh\ngithub.com/controlplaneio/simulator-standalone/pkg/util.Run\n\t/go/src/github.com/controlpla
neio/simulator-standalone/pkg/util/run.go:58\ngithub.com/controlplaneio/simulator-standalone/pkg/simulator.Perturb\n\t/go/src/github.com/controlplaneio/simulator-standalone/pkg/simulator/perturb.go:76\ngith
ub.com/controlplaneio/simulator-standalone/pkg/simulator.Launch\n\t/go/src/github.com/controlplaneio/simulator-standalone/pkg/simulator/launch.go:62\ngithub.com/controlplaneio/simulator-standalone/cmd.newSc
enarioLaunchCommand.func1\n\t/go/src/github.com/controlplaneio/simulator-standalone/cmd/scenario.go:49\ngithub.com/spf13/cobra.(*Command).execute\n\t/home/build/go/pkg/mod/github.com/spf13/[email protected]/comm
and.go:826\ngithub.com/spf13/cobra.(*Command).ExecuteC\n\t/home/build/go/pkg/mod/github.com/spf13/[email protected]/command.go:914\ngithub.com/spf13/cobra.(*Command).Execute\n\t/home/build/go/pkg/mod/github.com/
spf13/[email protected]/command.go:864\ngithub.com/controlplaneio/simulator-standalone/cmd.Execute\n\t/go/src/github.com/controlplaneio/simulator-standalone/cmd/root.go:109\nmain.main\n\t/go/src/github.com/contr
olplaneio/simulator-standalone/main.go:13\nruntime.main\n\t/usr/lib/go-1.11/src/runtime/proc.go:201\nruntime.goexit\n\t/usr/lib/go-1.11/src/runtime/asm_amd64.s:1333\nError running perturb with simulator.Per
turbOptions{Bastion:net.IP{0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0xff, 0xff, 0x3, 0x9, 0x76, 0xd3}, Master:net.IP{0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0xff, 0xff, 0xac, 0x1f, 0x2, 0
x25}, Slaves:[]net.IP{net.IP{0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0xff, 0xff, 0xac, 0x1f, 0x2, 0x33}, net.IP{0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0xff, 0xff, 0xac, 0x1f, 0x2, 0x2b}
}, ScenarioName:\"node-shock-tactics\"}\ngithub.com/controlplaneio/simulator-standalone/pkg/simulator.Launch\n\t/go/src/github.com/controlplaneio/simulator-standalone/pkg/simulator/launch.go:64\ngithub.com/
controlplaneio/simulator-standalone/cmd.newScenarioLaunchCommand.func1\n\t/go/src/github.com/controlplaneio/simulator-standalone/cmd/scenario.go:49\ngithub.com/spf13/cobra.(*Command).execute\n\t/home/build/
go/pkg/mod/github.com/spf13/[email protected]/command.go:826\ngithub.com/spf13/cobra.(*Command).ExecuteC\n\t/home/build/go/pkg/mod/github.com/spf13/[email protected]/command.go:914\ngithub.com/spf13/cobra.(*Command).
Execute\n\t/home/build/go/pkg/mod/github.com/spf13/[email protected]/command.go:864\ngithub.com/controlplaneio/simulator-standalone/cmd.Execute\n\t/go/src/github.com/controlplaneio/simulator-standalone/cmd/root.

go:109\nmain.main\n\t/go/src/github.com/controlplaneio/simulator-standalone/main.go:13\nruntime.main\n\t/usr/lib/go-1.11/src/runtime/proc.go:201\nruntime.goexit\n\t/usr/lib/go-1.11/src/runtime/asm_amd64.s:1
333"}
github.com/controlplaneio/simulator-standalone/cmd.newScenarioLaunchCommand.func1
        /go/src/github.com/controlplaneio/simulator-standalone/cmd/scenario.go:54
github.com/spf13/cobra.(*Command).execute
        /home/build/go/pkg/mod/github.com/spf13/[email protected]/command.go:826
github.com/spf13/cobra.(*Command).ExecuteC
        /home/build/go/pkg/mod/github.com/spf13/[email protected]/command.go:914
github.com/spf13/cobra.(*Command).Execute
        /home/build/go/pkg/mod/github.com/spf13/[email protected]/command.go:864
github.com/controlplaneio/simulator-standalone/cmd.Execute
        /go/src/github.com/controlplaneio/simulator-standalone/cmd/root.go:109
main.main
        /go/src/github.com/controlplaneio/simulator-standalone/main.go:13
runtime.main
        /usr/lib/go-1.11/src/runtime/proc.go:201
```


github.com/controlplaneio/simulator-standalone/cmd.newScenarioLaunchCommand.func1
        /go/src/github.com/controlplaneio/simulator-standalone/cmd/scenario.go:54
github.com/spf13/cobra.(*Command).execute
        /home/build/go/pkg/mod/github.com/spf13/[email protected]/command.go:826
github.com/spf13/cobra.(*Command).ExecuteC
        /home/build/go/pkg/mod/github.com/spf13/[email protected]/command.go:914
github.com/spf13/cobra.(*Command).Execute
        /home/build/go/pkg/mod/github.com/spf13/[email protected]/command.go:864
github.com/controlplaneio/simulator-standalone/cmd.Execute
        /go/src/github.com/controlplaneio/simulator-standalone/cmd/root.go:109
main.main
        /go/src/github.com/controlplaneio/simulator-standalone/main.go:13
runtime.main
        /usr/lib/go-1.11/src/runtime/proc.go:201


</details>

We should require all commits to PRs that will be merged into master should be signed. This will give us assurance commits are made by the account that is claiming to make them.

We can enable this option in the branch protection settings for master and we should do this as a matter of best practice.

Describe the bug
etcd is currently accessible by all nodes with the default kubeadm installation.

Expected behavior
Access to etcd should be restricted to the api server

Describe the issue
When we run a scenario, the output displays the commands for setting up the scenario, which gives excess information to the user, and could help them shortcut the scenario.

To Reproduce
Run any scenario and observe the output

Expected behavior
When running a scenario, there should be no information displayed which could unintentionally give information to help solve the scenario.

Feature statement:

Some of the scenarios require a starting point that enables them to have network visibility of the cluster, but not within it.

Describe the bug
Running simulator scenario launch without an interactive terminal leads to a failure without a relevant error message. I think the issue is to do with some of the bash terminal stuff that peturb does so #74 may help.

To Reproduce
Steps to reproduce the behavior:

1. Run:

docker run                                                          \
  -h launch                                                         \
  -v "${HOME}/.aws":/home/launch/.aws                \
  -v "${HOME}/.ssh":/home/launch/.ssh                         \
  -v "${HOME}/.kubesim":/home/launch/.kubesim                        \
  -e "AWS_ACCESS_KEY_ID"                                            \
  -e "AWS_SECRET_ACCESS_KEY"                                        \
  -e "AWS_REGION"                                                   \
  --rm --init  ${CONTAINER_NAME} -- bash -c  "simulator infra destroy; simulator infra create ; simulator scenario launch rbac-sangar"

Expected behavior
The scenario should provision correctly without failure.

Environment (please complete the following information):

• OS: Ubuntu 18.04
• Docker: 19.03.4
• Version of simulator: master-db0daa3180c5b7ae93b473ac40fc1236d8d2a00c

Describe the bug
When starting a run from no bucket and creating a bucket with a new name, you can not create infra.

Error message:
Error: Failed to get existing workspaces: S3 bucket does not exist.

To Reproduce
from host, delete simulator yaml.

$simulator init
$simulator infra create

Describe the bug
When you compile kubesim from scratch it currently takes over 30 mins to complete.

Even when you re-run make run this also take a very long time to complete. This appears to be caused by the large number of external libraries required.

For scenario: rbac-flanking-maneuver

The description contains a placeholder for the $MASTER_IP_ADDRESS, but this is not being resolved.

Please, see attached image.

Describe the bug
Deleting or overwriting the ~/.ssh/cp_simulator_* files makes the cluster permanently inaccessible. Attempting to to simulator ssh attack fails as the key is not valid. simulator infra create will reprovision the bastion keypair but the new key is not accepted by the bastion.

To Reproduce
Steps to reproduce the behavior:

1. simulator infra create
2. rm ~/.ssh/cp_simulator_*
3. simulator ssh attack
4. See error
5. simulator infra create
6. See new key be provisioned
7. simulator ssh attack
8. See same error as before

Expected behavior
I expect simulator to recover from a missing ssh key by reprovisioning the key or bastion with simulator infra create. simulator ssh attack should prompt the user to simulator infra create if the key does not work.

Environment (please complete the following information):

• OS: Ubuntu 18.04
• Docker: 19.03.4
• Version of simulator: master-7a2440a96e83dbb9dba6631b664fcb01cb7516a6

• divide by type (i.e. network, node, etc)
• submark by difficulty

Currently there is a KIND POC in /kind in the kind branch but needs to be updated for recent changes to simulator and to be fleshed out into a full feature.

Quite a few scripts in the simulator use #!/usr/bin/env bash. This has security implications and should be replaced with full path to bash, #!/bin/bash.

Describe the bug
When perturb exit with a non-zero,the simulator produces excessive errors which means the error statement from perturb can be lost

To Reproduce
Steps to reproduce the behavior:

1. Use branch verify-run-once
2. start simulator as usual, init and create infracture
3. deploy a scenario
4. depoy another (or same) scenario

Expected behavior
To be able view the error exit statement from perturb clearly

Example output

First line is a valid error from perturb, remainder if the resultant output from simulator

[2019-12-02T12:01:37+0000] ./perturb.sh: Perturbed has been previously run on this cluster
1.5752880979839957e+09	ERROR	cmd/scenario.go:54	Error launching scenario	{"error": "Error running perturb with simulator.PerturbOptions{Bastion:net.IP{0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0xff, 0xff, 0x22, 0xf4, 0x7a, 0x8b}, Master:net.IP{0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0xff, 0xff, 0xac, 0x1f, 0x2, 0xfd}, Slaves:[]net.IP{net.IP{0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0xff, 0xff, 0xac, 0x1f, 0x2, 0x64}, net.IP{0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0xff, 0xff, 0xac, 0x1f, 0x2, 0x47}}, ScenarioName:\"etcd-inverted-wedge\"}: Error waiting for child process ./perturb.sh: exit status 3", "errorVerbose": "exit status 3\nError waiting for child process ./perturb.sh\ngithub.com/controlplaneio/simulator-standalone/pkg/util.Run\n\t/go/src/github.com/controlplaneio/simulator-standalone/pkg/util/run.go:58\ngithub.com/controlplaneio/simulator-standalone/pkg/simulator.Perturb\n\t/go/src/github.com/controlplaneio/simulator-standalone/pkg/simulator/perturb.go:76\ngithub.com/controlplaneio/simulator-standalone/pkg/simulator.Launch\n\t/go/src/github.com/controlplaneio/simulator-standalone/pkg/simulator/launch.go:62\ngithub.com/controlplaneio/simulator-standalone/cmd.newScenarioLaunchCommand.func1\n\t/go/src/github.com/controlplaneio/simulator-standalone/cmd/scenario.go:49\ngithub.com/spf13/cobra.(*Command).execute\n\t/home/build/go/pkg/mod/github.com/spf13/[email protected]/command.go:826\ngithub.com/spf13/cobra.(*Command).ExecuteC\n\t/home/build/go/pkg/mod/github.com/spf13/[email protected]/command.go:914\ngithub.com/spf13/cobra.(*Command).Execute\n\t/home/build/go/pkg/mod/github.com/spf13/[email protected]/command.go:864\ngithub.com/controlplaneio/simulator-standalone/cmd.Execute\n\t/go/src/github.com/controlplaneio/simulator-standalone/cmd/root.go:109\nmain.main\n\t/go/src/github.com/controlplaneio/simulator-standalone/main.go:13\nruntime.main\n\t/usr/lib/go-1.11/src/runtime/proc.go:201\nruntime.goexit\n\t/usr/lib/go-1.11/src/runtime/asm_amd64.s:1333\nError running perturb with simulator.PerturbOptions{Bastion:net.IP{0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0xff, 0xff, 0x22, 0xf4, 0x7a, 0x8b}, Master:net.IP{0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0xff, 0xff, 0xac, 0x1f, 0x2, 0xfd}, Slaves:[]net.IP{net.IP{0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0xff, 0xff, 0xac, 0x1f, 0x2, 0x64}, net.IP{0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0xff, 0xff, 0xac, 0x1f, 0x2, 0x47}}, ScenarioName:\"etcd-inverted-wedge\"}\ngithub.com/controlplaneio/simulator-standalone/pkg/simulator.Launch\n\t/go/src/github.com/controlplaneio/simulator-standalone/pkg/simulator/launch.go:64\ngithub.com/controlplaneio/simulator-standalone/cmd.newScenarioLaunchCommand.func1\n\t/go/src/github.com/controlplaneio/simulator-standalone/cmd/scenario.go:49\ngithub.com/spf13/cobra.(*Command).execute\n\t/home/build/go/pkg/mod/github.com/spf13/[email protected]/command.go:826\ngithub.com/spf13/cobra.(*Command).ExecuteC\n\t/home/build/go/pkg/mod/github.com/spf13/[email protected]/command.go:914\ngithub.com/spf13/cobra.(*Command).Execute\n\t/home/build/go/pkg/mod/github.com/spf13/[email protected]/command.go:864\ngithub.com/controlplaneio/simulator-standalone/cmd.Execute\n\t/go/src/github.com/controlplaneio/simulator-standalone/cmd/root.go:109\nmain.main\n\t/go/src/github.com/controlplaneio/simulator-standalone/main.go:13\nruntime.main\n\t/usr/lib/go-1.11/src/runtime/proc.go:201\nruntime.goexit\n\t/usr/lib/go-1.11/src/runtime/asm_amd64.s:1333"}
github.com/controlplaneio/simulator-standalone/cmd.newScenarioLaunchCommand.func1
	/go/src/github.com/controlplaneio/simulator-standalone/cmd/scenario.go:54
github.com/spf13/cobra.(*Command).execute
	/home/build/go/pkg/mod/github.com/spf13/[email protected]/command.go:826
github.com/spf13/cobra.(*Command).ExecuteC
	/home/build/go/pkg/mod/github.com/spf13/[email protected]/command.go:914
github.com/spf13/cobra.(*Command).Execute
	/home/build/go/pkg/mod/github.com/spf13/[email protected]/command.go:864
github.com/controlplaneio/simulator-standalone/cmd.Execute
	/go/src/github.com/controlplaneio/simulator-standalone/cmd/root.go:109
main.main
	/go/src/github.com/controlplaneio/simulator-standalone/main.go:13
runtime.main
	/usr/lib/go-1.11/src/runtime/proc.go:201

Go 1.13 added new Unwrap and errors.Is ans errors.As helpers based on the popularity of libraries like github.com/pkg/errors which we currently use. We should switch to the new standard library way of doing things and remove our dependency on the errors package. This is also a good opportunity to review the error messages for usability and consistency.

Background reading on go 1.13 errors here: https://blog.golang.org/go1.13-errors

Feature statement
At the moment cards are populated into the board manually when an issue is raised.

Proposed approach
There are a few actions bots that could be used to populate cards and move cards around based on the status of PRs or issues. Investigate the feasibility/ whether they will work for us.

• Simulator environments in the same account need to be uniquely identifiable
• The bucket name used to create each environment may be a good tag to use

Describe the bug
CAP_NET_RAW is enabled by default for pods. If an attacker manages to compromise a pod, they could potentially take advantage of this capability to perform network attacks on other pods running on the same node

Expected behavior
Should this be enabled by default?

https://blog.trailofbits.com/2019/11/07/attacking-go-vr-ttps/

In order to avoid introducing detectable security vulnerabilities in golang code, we should automate running at least:

• go vet
• staticcheck

And consider adding:

• Fuzz testing
• Property based black box testing a la quickcheck

Inform the use in the README how we use external services to detect their IP address and then lock the AWS K8s cluster to access from that IP. Include instructions of what to expect and how to reconfigure if they are e.g. on a laptop and move from a coffee shop back to the office after creating infra.

So people can't cat it from the attack container when the scenario starts.

Feature statement:

All the scenarios require a starting point that enables the user to begin working on the scenario.

Timestamps in simulator are currently displayed as:

1.5760591043023489e+09

This is very unreadable to a human as we would have to count 9 characters to place the decimal in the correct position to convert back to a human readable time like so:

1576059104.3023489

Also as can be seen above the e+09 is completely redundant and should be removed. We display more than 10 characters so we have no reason to use a shorthand. Especially when our precision is purportedly 0.1 microseconds.

We should use a human readable date format instead. Or at least remove the e+09 so users can quickly copy and paste the timestamp into a converter.

The output from 'simulator scenario list' is messy and needs formatting.

l

Suggest removing the epoch numbers, and the go code lines

When executing:

simulator infra destroy

Output:

`launch@launch:/app$ simulator infra destroy
1.572539731390471e+09 INFO simulator/terraform.go:50 Running terraform init
Initializing modules...

• Ami in ../../modules/AWS/Ami
• Bastion in ../../modules/AWS/Bastion
• InternalNode in ../../modules/AWS/InternalNode
• Kubernetes in ../../modules/AWS/Kubernetes
• Networking in ../../modules/AWS/Networking
• S3Storage in ../../modules/AWS/S3Storage
• SecurityGroups in ../../modules/AWS/SecurityGroups
• SshKey in ../../modules/AWS/SshKey

Initializing the backend...

Successfully configured the backend "s3"! Terraform will automatically
use this backend unless the backend configuration changes.

Error: Failed to get existing workspaces: InvalidBucketName: The specified bucket is not valid.
status code: 400, request id: 820FC0434E1F175B, host id: TA2hUu6hFa9Vt0ZZi3nLnHMz6mfaHpL0z/nNs/qIQLWfox08SqYmOFtwnkAkTO0lZQOLj8bamfI=

1.57253973303412e+09 ERROR cmd/infra.go:96 Error destroying infrastructure{"error": "Error initialising: Error initialising terraform: Error waiting for child process terraform: exit status 1", "errorVerbose": "exit status 1\nError waiting for child process terraform\ngithub.com/controlplaneio/simulator-standalone/pkg/util.Run\n\t/go/src/github.com/controlplaneio/simulator-standalone/pkg/util/run.go:58\ngithub.com/controlplaneio/simulator-standalone/pkg/simulator.Terraform\n\t/go/src/github.com/controlplaneio/simulator-standalone/pkg/simulator/terraform.go:42\ngithub.com/controlplaneio/simulator-standalone/pkg/simulator.InitIfNeeded\n\t/go/src/github.com/controlplaneio/simulator-standalone/pkg/simulator/terraform.go:51\ngithub.com/controlplaneio/simulator-standalone/pkg/simulator.Destroy\n\t/go/src/github.com/controlplaneio/simulator-standalone/pkg/simulator/terraform.go:137\ngithub.com/controlplaneio/simulator-standalone/cmd.newDestroyCommand.func1\n\t/go/src/github.com/controlplaneio/simulator-standalone/cmd/infra.go:94\ngithub.com/spf13/cobra.(*Command).execute\n\t/home/build/go/pkg/mod/github.com/spf13/[email protected]/command.go:826\ngithub.com/spf13/cobra.(*Command).ExecuteC\n\t/home/build/go/pkg/mod/github.com/spf13/[email protected]/command.go:914\ngithub.com/spf13/cobra.(*Command).Execute\n\t/home/build/go/pkg/mod/github.com/spf13/[email protected]/command.go:864\ngithub.com/controlplaneio/simulator-standalone/cmd.Execute\n\t/go/src/github.com/controlplaneio/simulator-standalone/cmd/root.go:105\nmain.main\n\t/go/src/github.com/controlplaneio/simulator-standalone/main.go:13\nruntime.main\n\t/usr/lib/go-1.11/src/runtime/proc.go:201\nruntime.goexit\n\t/usr/lib/go-1.11/src/runtime/asm_amd64.s:1333\nError initialising terraform\ngithub.com/controlplaneio/simulator-standalone/pkg/simulator.InitIfNeeded\n\t/go/src/github.com/controlplaneio/simulator-standalone/pkg/simulator/terraform.go:53\ngithub.com/controlplaneio/simulator-standalone/pkg/simulator.Destroy\n\t/go/src/github.com/controlplaneio/simulator-standalone/pkg/simulator/terraform.go:137\ngithub.com/controlplaneio/simulator-standalone/cmd.newDestroyCommand.func1\n\t/go/src/github.com/controlplaneio/simulator-standalone/cmd/infra.go:94\ngithub.com/spf13/cobra.(*Command).execute\n\t/home/build/go/pkg/mod/github.com/spf13/[email protected]/command.go:826\ngithub.com/spf13/cobra.(*Command).ExecuteC\n\t/home/build/go/pkg/mod/github.com/spf13/[email protected]/command.go:914\ngithub.com/spf13/cobra.(*Command).Execute\n\t/home/build/go/pkg/mod/github.com/spf13/[email protected]/command.go:864\ngithub.com/controlplaneio/simulator-standalone/cmd.Execute\n\t/go/src/github.com/controlplaneio/simulator-standalone/cmd/root.go:105\nmain.main\n\t/go/src/github.com/controlplaneio/simulator-standalone/main.go:13\nruntime.main\n\t/usr/lib/go-1.11/src/runtime/proc.go:201\nruntime.goexit\n\t/usr/lib/go-1.11/src/runtime/asm_amd64.s:1333\nError initialising\ngithub.com/controlplaneio/simulator-standalone/pkg/simulator.Destroy\n\t/go/src/github.com/controlplaneio/simulator-standalone/pkg/simulator/terraform.go:139\ngithub.com/controlplaneio/simulator-standalone/cmd.newDestroyCommand.func1\n\t/go/src/github.com/controlplaneio/simulator-standalone/cmd/infra.go:94\ngithub.com/spf13/cobra.(*Command).execute\n\t/home/build/go/pkg/mod/github.com/spf13/[email protected]/command.go:826\ngithub.com/spf13/cobra.(*Command).ExecuteC\n\t/home/build/go/pkg/mod/github.com/spf13/[email protected]/command.go:914\ngithub.com/spf13/cobra.(*Command).Execute\n\t/home/build/go/pkg/mod/github.com/spf13/[email protected]/command.go:864\ngithub.com/controlplaneio/simulator-standalone/cmd.Execute\n\t/go/src/github.com/controlplaneio/simulator-standalone/cmd/root.go:105\nmain.main\n\t/go/src/github.com/controlplaneio/simulator-standalone/main.go:13\nruntime.main\n\t/usr/lib/go-1.11/src/runtime/proc.go:201\nruntime.goexit\n\t/usr/lib/go-1.11/src/runtime/asm_amd64.s:1333"}
github.com/controlplaneio/simulator-standalone/cmd.newDestroyCommand.func1
/go/src/github.com/controlplaneio/simulator-standalone/cmd/infra.go:96
github.com/spf13/cobra.(*Command).execute
/home/build/go/pkg/mod/github.com/spf13/[email protected]/command.go:826
github.com/spf13/cobra.(*Command).ExecuteC
/home/build/go/pkg/mod/github.com/spf13/[email protected]/command.go:914
github.com/spf13/cobra.(*Command).Execute
/home/build/go/pkg/mod/github.com/spf13/[email protected]/command.go:864
github.com/controlplaneio/simulator-standalone/cmd.Execute
/go/src/github.com/controlplaneio/simulator-standalone/cmd/root.go:105
main.main
/go/src/github.com/controlplaneio/simulator-standalone/main.go:13
runtime.main
/usr/lib/go-1.11/src/runtime/proc.go:201
Error: Error initialising: Error initialising terraform: Error waiting for child process terraform: exit status 1
Usage:
simulator infra destroy [flags]

Flags:
-h, --help help for destroy

Global Flags:
-c, --config-file string Path to the simulator config file
-l, --loglevel string Level of detail in output logging (default "info")
-s, --scenarios-dir string Path to a directory containing a scenario manifest (default "./simulation-scripts")
-b, --state-bucket string The name of the s3 bucket to use for remote-state. Must be globally unique
-t, --tf-dir string Path to a directory containing the infrastructure scripts (default "./terraform/deployments/AWS")

Error initialising: Error initialising terraform: Error waiting for child process terraform: exit status 1`

Verification:

simulator config get state-bucket

launch@launch:/app$ simulator config get state-bucket 1.5725399745775201e+09 INFO cmd/config.go:20 state-bucket = agarcia-scenarios-testing\n

Verification:

simulator infra status

`launch@launch:/app$ simulator infra status
1.5725400588509696e+09 INFO simulator/terraform.go:50 Running terraform init
Initializing modules...

Initializing the backend...

Error: Failed to get existing workspaces: InvalidBucketName: The specified bucket is not valid.
status code: 400, request id: F4A02C55E37B353F, host id: NWtdZPxehMsDzcauFJ7MmBUkHZZGNKFOtebH9Ipjj1qNdCiIWqkwN4/hso0PtNbcdwKTqrdPZnU=

1.5725400604029083e+09 ERROR cmd/infra.go:62 Error getting status of infrastructure {"error": "Error initialising: Error initialising terraform: Error waiting for child process terraform: exit status 1", "errorVerbose": "exit status 1\nError waiting for child process terraform\ngithub.com/controlplaneio/simulator-standalone/pkg/util.Run\n\t/go/src/github.com/controlplaneio/simulator-standalone/pkg/util/run.go:58\ngithub.com/controlplaneio/simulator-standalone/pkg/simulator.Terraform\n\t/go/src/github.com/controlplaneio/simulator-standalone/pkg/simulator/terraform.go:42\ngithub.com/controlplaneio/simulator-standalone/pkg/simulator.InitIfNeeded\n\t/go/src/github.com/controlplaneio/simulator-standalone/pkg/simulator/terraform.go:51\ngithub.com/controlplaneio/simulator-standalone/pkg/simulator.Status\n\t/go/src/github.com/controlplaneio/simulator-standalone/pkg/simulator/terraform.go:113\ngithub.com/controlplaneio/simulator-standalone/cmd.newStatusCommand.func1\n\t/go/src/github.com/controlplaneio/simulator-standalone/cmd/infra.go:60\ngithub.com/spf13/cobra.(*Command).execute\n\t/home/build/go/pkg/mod/github.com/spf13/[email protected]/command.go:826\ngithub.com/spf13/cobra.(*Command).ExecuteC\n\t/home/build/go/pkg/mod/github.com/spf13/[email protected]/command.go:914\ngithub.com/spf13/cobra.(*Command).Execute\n\t/home/build/go/pkg/mod/github.com/spf13/[email protected]/command.go:864\ngithub.com/controlplaneio/simulator-standalone/cmd.Execute\n\t/go/src/github.com/controlplaneio/simulator-standalone/cmd/root.go:105\nmain.main\n\t/go/src/github.com/controlplaneio/simulator-standalone/main.go:13\nruntime.main\n\t/usr/lib/go-1.11/src/runtime/proc.go:201\nruntime.goexit\n\t/usr/lib/go-1.11/src/runtime/asm_amd64.s:1333\nError initialising terraform\ngithub.com/controlplaneio/simulator-standalone/pkg/simulator.InitIfNeeded\n\t/go/src/github.com/controlplaneio/simulator-standalone/pkg/simulator/terraform.go:53\ngithub.com/controlplaneio/simulator-standalone/pkg/simulator.Status\n\t/go/src/github.com/controlplaneio/simulator-standalone/pkg/simulator/terraform.go:113\ngithub.com/controlplaneio/simulator-standalone/cmd.newStatusCommand.func1\n\t/go/src/github.com/controlplaneio/simulator-standalone/cmd/infra.go:60\ngithub.com/spf13/cobra.(*Command).execute\n\t/home/build/go/pkg/mod/github.com/spf13/[email protected]/command.go:826\ngithub.com/spf13/cobra.(*Command).ExecuteC\n\t/home/build/go/pkg/mod/github.com/spf13/[email protected]/command.go:914\ngithub.com/spf13/cobra.(*Command).Execute\n\t/home/build/go/pkg/mod/github.com/spf13/[email protected]/command.go:864\ngithub.com/controlplaneio/simulator-standalone/cmd.Execute\n\t/go/src/github.com/controlplaneio/simulator-standalone/cmd/root.go:105\nmain.main\n\t/go/src/github.com/controlplaneio/simulator-standalone/main.go:13\nruntime.main\n\t/usr/lib/go-1.11/src/runtime/proc.go:201\nruntime.goexit\n\t/usr/lib/go-1.11/src/runtime/asm_amd64.s:1333\nError initialising\ngithub.com/controlplaneio/simulator-standalone/pkg/simulator.Status\n\t/go/src/github.com/controlplaneio/simulator-standalone/pkg/simulator/terraform.go:115\ngithub.com/controlplaneio/simulator-standalone/cmd.newStatusCommand.func1\n\t/go/src/github.com/controlplaneio/simulator-standalone/cmd/infra.go:60\ngithub.com/spf13/cobra.(*Command).execute\n\t/home/build/go/pkg/mod/github.com/spf13/[email protected]/command.go:826\ngithub.com/spf13/cobra.(*Command).ExecuteC\n\t/home/build/go/pkg/mod/github.com/spf13/[email protected]/command.go:914\ngithub.com/spf13/cobra.(*Command).Execute\n\t/home/build/go/pkg/mod/github.com/spf13/[email protected]/command.go:864\ngithub.com/controlplaneio/simulator-standalone/cmd.Execute\n\t/go/src/github.com/controlplaneio/simulator-standalone/cmd/root.go:105\nmain.main\n\t/go/src/github.com/controlplaneio/simulator-standalone/main.go:13\nruntime.main\n\t/usr/lib/go-1.11/src/runtime/proc.go:201\nruntime.goexit\n\t/usr/lib/go-1.11/src/runtime/asm_amd64.s:1333"}
github.com/controlplaneio/simulator-standalone/cmd.newStatusCommand.func1
/go/src/github.com/controlplaneio/simulator-standalone/cmd/infra.go:62
github.com/spf13/cobra.(*Command).execute
/home/build/go/pkg/mod/github.com/spf13/[email protected]/command.go:826
github.com/spf13/cobra.(*Command).ExecuteC
/home/build/go/pkg/mod/github.com/spf13/[email protected]/command.go:914
github.com/spf13/cobra.(*Command).Execute
/home/build/go/pkg/mod/github.com/spf13/[email protected]/command.go:864
github.com/controlplaneio/simulator-standalone/cmd.Execute
/go/src/github.com/controlplaneio/simulator-standalone/cmd/root.go:105
main.main
/go/src/github.com/controlplaneio/simulator-standalone/main.go:13
runtime.main
/usr/lib/go-1.11/src/runtime/proc.go:201
Error: Error initialising: Error initialising terraform: Error waiting for child process terraform: exit status 1
Usage:
simulator infra status [flags]

Flags:
-h, --help help for status

Global Flags:
-c, --config-file string Path to the simulator config file
-l, --loglevel string Level of detail in output logging (default "info")
-s, --scenarios-dir string Path to a directory containing a scenario manifest (default "./simulation-scripts")
-b, --state-bucket string The name of the s3 bucket to use for remote-state. Must be globally unique
-t, --tf-dir string Path to a directory containing the infrastructure scripts (default "./terraform/deployments/AWS")

Error initialising: Error initialising terraform: Error waiting for child process terraform: exit status 1`

Feature statement
Presently perturb will run against a clustered that has already been perturbed, leading to unexpected behaviour for the scenario in question.

Proposed approach
There are 2 possible approaches:

1. Ensure perturb validates the state of the cluster before applying scenrio
2. Use approach outlined in issue #70

Describe the bug
When running some scenarios (secret-high-ground, possibly others) they hang and timeout on waiting for pods to be initialised.

Additional context (optional)
Was replicated by myself and Jon Knox on the same network. Might be worth testing on different network.

Describe the bug
Editing providers.tf prevents using a read only mount in the container. This prevents the simulator infra create command from working as it edits a file on a read-only filesystem. Issue found when working #100

To Reproduce
Steps to reproduce the behavior:

1. Mount a local terraform dir into the container using -v $(shell pwd)/terraform:/app/terraform
2. Run simulator infra create

Expected behavior
We should be able to mount this directory into the container as read-only to facilitate local workflow and prevent accidental edits going back to the host fs.

These links may help:
tf docs
Github issue

Describe the bug
perturb is producing the following error:

mesg: ttyname failed: Inappropriate ioctl for device

This is not causing any issues, but needs to be addressed

To Reproduce
Steps to reproduce the behavior:

1. Start launch container
2. Create infrastructure
3. Launch scenario
4. See error on output

Expected behavior
Only produce usable errors

Environment (please complete the following information):

• OS: MacOS 10.15
• Docker: 2.1.0.3(38240)
• Version of simulator: master

Feature statement
Currently the terraform cloud init configuration installs the latest version of kubernetes (kubeadm, kubectl etc).

Simulator should be able to specify the version of Kubernetes that is to be installed for the scenario, defaulting to the latest if none is defined.

Issue

At present simulator first stands up the infrastructure, which includes installing kubernetes (via kubeadm), using a cloud-init script on the nodes. Then the scenario is applied. To specify a version of kubernetes this workflow is incorrect.

Proposed approach

Change the workflow order from simulator infra create and simulator scenario launch to just simulator scenario launch. This single command will action:

• Verify if existing infrastructure exists, destroy if it does (maybe with a verification input from user first)
• Pass Kubernetes version to Terraform (with a Terraform default is none supplied)
• Terraform creates node cloud-init yaml files from jinja templates to apply kubernetes version
• Create infrastructure
• Deploy scenario

Feature statement
State the current approach (if exists), what the feature would provide, and the expected outcome.

Issue
If this addresses/or enhances a issue, state what this issue is that this feature will resolve

Proposed approach
Outline at a high level how this feature would be implemented. If many steps break these down into a bullet list.

When running make help the following output is received:

setup-dev             Initialise simulation tree with git hooks
devtools-deps: # Install devtools dependencies
devtools              Install devtools
test-devtools: # Run all the unit tests for the devtools
reset                 Clean up files left over by simulator
validate-requirements  Verify all requirements are met
previous-tag:
release-tag:
gpg-preflight:
run                   Run the simulator - the build stage of the container runs all the cli tests
docker-build-no-cache  Builds the launch container
docker-build          Builds the launch container
docker-test           Run the tests
dep                   Install dependencies for other targets
build                 Run golang build for the CLI program
is-in-launch-container  checks you are running in the launch container
test                  run all tests except goss tests
test-acceptance       Run tcl acceptance tests for the CLI program
test-smoke            Run expect smoke test to check happy path works end-to-end
test-unit             Run golang unit tests for the CLI program
test-cleanup          cleans up automated test artefacts if e.g. you ctrl-c abort a test run
test-coverage         Run golang unit tests with coverage and opens a browser with the results
docs                  Generate documentation
release: gpg-preflight previous-tag release-tag docker-test docker-build build

Some target help is missing completely, e.g previous-tag, some is meaningless, e.g. release and some have trailing #, e.g. test-devtools

In the line https://github.com/kubernetes-simulator/simulator/blob/master/attack/scripts/bashrc#L12, there is the word perimiter where it should be perimeter.

We should think about solving these problems:

• Why isn't this automatic?
• How to improve the templating so that bash can be included inline without throwing the templating system - it uses bash var substitution which is limiting
• How to reduce commit noise with doc generation
• godocdown is now broken .. it is unmaintained and doesn't use gomodules and a downstream dependency has disappeared causing make dep to fail. As much as I dislike godoc, I think we should remove the API docs from the repo and lean on using godoc.org for the golang docs

Inform the use in the README how we use external services to detect their IP address and then lock the AWS K8s cluster to access from that IP. Include instructions of what to expect and how to reconfigure if they are e.g. on a laptop and move from a coffee shop back to the office after creating infra.

Feature statement
We are missing a template for a feature request, this should be added

Issue
At present you have to either delete the Issue or blank template when requesting a feature, which will lead to various methods of creating the request

Proposed Approach
Create a template for feature requests

Feature statement
In the Current approach, during make run the container is built and packages up the current state of the terraform and scenario directories. Any changes made to these directories are not reflected in the container so developing scenarios or terraform requires constantly exiting and rebuilding the container.

Proposed approach
make run should mount the terraform and scenario directories so changes made to them can be applied to cluster without exiting or rebuilding the container.

Currently getting the following error when running make release

spawn simulator version

1.5736369583883626e+09  INFO     cmd/version.go:26       version v0.10

1.5736369583887901e+09  INFO     cmd/version.go:27       git commit c8472a90699d9572c64eceac5652141db0646577

1.5736369583898497e+09  INFO     cmd/version.go:28       build date 2019-11-13T09:18:55Z

++++ version_prints_ok PASSED
Sourced 0 Test Files.

Tests ended at Wed Nov 13 09:22:38 UTC 2019
run-tests.tcl:  Total   5       Passed  3       Skipped 0       Failed  2
Sourced 4 Test Files.
Files with failing tests: init.test
make: *** [docker-test] Error 1

Possible down to AWS_SHARED_CREDENTIALS_FILE variable?

At the moment we can template out environment variables available to the attack container in the challenge.txt. This includes Node/Master IPs but needs to be expanded to allow the templating of dynamic pod names/IPs.

Feature statement
State the current approach (if exists), what the feature would provide, and the expected outcome.

Issue
If this addresses/or enhances a issue, state what this issue is that this feature will resolve

Proposed approach
Outline at a high level how this feature would be implemented. If many steps break these down into a bullet list.

Although there is limited help provided via make help, the actual flow can be confusing and therefore needs to be documented.