cyberark / summon Goto Github PK

As of 0.6.9 subprocess loading no longer works on Windows as a result of switching to execve. We'll need to add a case for Windows to use the previous loading behavior.

I am currently evaluating summon to be a solution for secrets inside docker. I see there is an example of injecting it into the ENV by wrapping the docker command with a summon command and I am currently running this to see how it behaves.

I don't think this will work with my longer term goal which is to interact with docker through docker APIs &&|| cli thereby bypassing the summon wrapper type interface.

Essentially I need to be able to do something like:

docker run -v secrets:/secrets:ro --volume-driver summon alpine

and either have /secrets be a file of newline separated K:V pairs, a directory tree of /secrets/:key where the contents of the files are the corresponding values, somehow have it be a memory mapped file, or somehow be able to inject it via the --env-file flag.

I am currently prototyping some things, but I am curious to get the communities thoughts on this as well.

Thanks in advance!

We have seen this several times and it kills the process called from summon.

fatal error: runtime: out of memory
runtime stack:
runtime.throw(0x5b044f, 0x16)
/usr/local/go/src/runtime/panic.go:596 +0x95
runtime.sysMap(0xc49d500000, 0x7d0d0000, 0x0, 0x695738)
/usr/local/go/src/runtime/mem_linux.go:216 +0x1d0
runtime.(*mheap).sysAlloc(0x67d5c0, 0x7d0d0000, 0xc420247e18)
/usr/local/go/src/runtime/malloc.go:440 +0x374
runtime.(*mheap).grow(0x67d5c0, 0x3e868, 0x0)
/usr/local/go/src/runtime/mheap.go:774 +0x62
runtime.(*mheap).allocSpanLocked(0x67d5c0, 0x3e868, 0x44dc0a)
/usr/local/go/src/runtime/mheap.go:678 +0x44f
runtime.(*mheap).alloc_m(0x67d5c0, 0x3e868, 0x100000000, 0x0)
/usr/local/go/src/runtime/mheap.go:562 +0xe2
runtime.(*mheap).alloc.func1()
/usr/local/go/src/runtime/mheap.go:627 +0x4b
runtime.systemstack(0xc420247f10)
/usr/local/go/src/runtime/asm_amd64.s:343 +0xab
runtime.(*mheap).alloc(0x67d5c0, 0x3e868, 0x10100000000, 0x42f7c1)
/usr/local/go/src/runtime/mheap.go:628 +0xa0
runtime.largeAlloc(0x7d0cfd77, 0xc42001d301, 0xc4203f2680)
/usr/local/go/src/runtime/malloc.go:807 +0x93
runtime.mallocgc.func1()
/usr/local/go/src/runtime/malloc.go:702 +0x3e
runtime.systemstack(0xc42001c000)
/usr/local/go/src/runtime/asm_amd64.s:327 +0x79
runtime.mstart()
/usr/local/go/src/runtime/proc.go:1132
goroutine 143 [running]:
runtime.systemstack_switch()
/usr/local/go/src/runtime/asm_amd64.s:281 fp=0xc42045fc60 sp=0xc42045fc58
runtime.mallocgc(0x7d0cfd77, 0x574520, 0xc420416001, 0x0)
/usr/local/go/src/runtime/malloc.go:703 +0x930 fp=0xc42045fd00 sp=0xc42045fc60
runtime.makeslice(0x574520, 0x7d0cfd77, 0x7d0cfd77, 0x8ed, 0x8000, 0x0)
/usr/local/go/src/runtime/slice.go:54 +0x7b fp=0xc42045fd50 sp=0xc42045fd00
bytes.makeSlice(0x7d0cfd77, 0x0, 0x0, 0x0)
/usr/local/go/src/bytes/buffer.go:201 +0x77 fp=0xc42045fd90 sp=0xc42045fd50
bytes.(*Buffer).grow(0xc42040dce0, 0x8ed, 0x0)
/usr/local/go/src/bytes/buffer.go:109 +0x177 fp=0xc42045fde0 sp=0xc42045fd90
bytes.(*Buffer).Write(0xc42040dce0, 0xc4204a8000, 0x8ed, 0x8000, 0x8ed, 0x0, 0x0)
/usr/local/go/src/bytes/buffer.go:137 +0x41 fp=0xc42045fe10 sp=0xc42045fde0
io.(*multiWriter).Write(0xc4203c52e0, 0xc4204a8000, 0x8ed, 0x8000, 0x8ed, 0x0, 0x0)
/usr/local/go/src/io/multi.go:60 +0x72 fp=0xc42045fe70 sp=0xc42045fe10
io.copyBuffer(0x664420, 0xc4203c52e0, 0x6644a0, 0xc4200a84e0, 0xc4204a8000, 0x8000, 0x8000, 0x0, 0x0, 0x0)
/usr/local/go/src/io/io.go:392 +0x20b fp=0xc42045fed8 sp=0xc42045fe70
io.Copy(0x664420, 0xc4203c52e0, 0x6644a0, 0xc4200a84e0, 0x0, 0x0, 0x0)
/usr/local/go/src/io/io.go:360 +0x68 fp=0xc42045ff38 sp=0xc42045fed8
os/exec.(*Cmd).writerDescriptor.func1(0x0, 0x0)
/usr/local/go/src/os/exec/exec.go:254 +0x4d fp=0xc42045ff98 sp=0xc42045ff38
os/exec.(*Cmd).Start.func1(0xc4200a2420, 0xc4203c5340)
/usr/local/go/src/os/exec/exec.go:371 +0x27 fp=0xc42045ffd0 sp=0xc42045ff98
runtime.goexit()
/usr/local/go/src/runtime/asm_amd64.s:2197 +0x1 fp=0xc42045ffd8 sp=0xc42045ffd0
created by os/exec.(*Cmd).Start
/usr/local/go/src/os/exec/exec.go:372 +0x4e4
goroutine 1 [syscall, 2358 minutes]:
syscall.Syscall6(0xf7, 0x1, 0x12d, 0xc4200b1688, 0x1000004, 0x0, 0x0, 0x7f24929e74b0, 0x0, 0xc4203c5320)
/usr/local/go/src/syscall/asm_linux_amd64.s:44 +0x5
os.(*Process).blockUntilWaitable(0xc420417bc0, 0xc4200a2420, 0xc4203c5340, 0x2)
/usr/local/go/src/os/wait_waitid.go:28 +0xa5
os.(*Process).wait(0xc420417bc0, 0x0, 0xc4200a84f8, 0xc4203c5220)
/usr/local/go/src/os/exec_unix.go:22 +0x4d
os.(*Process).Wait(0xc420417bc0, 0x0, 0x0, 0x5b7490)
/usr/local/go/src/os/exec.go:115 +0x2b
os/exec.(*Cmd).Wait(0xc4200a2420, 0x0, 0x0)
/usr/local/go/src/os/exec/exec.go:435 +0x62
os/exec.(*Cmd).Run(0xc4200a2420, 0x2, 0x2)
/usr/local/go/src/os/exec/exec.go:280 +0x5c
github.com/conjurinc/summon/command.runSubcommand(0xc42000e140, 0x2, 0x2, 0xc42006a480, 0x42, 0x48, 0x40, 0x30, 0x0, 0xc4200c6380)
/go/src/github.com/conjurinc/summon/command/subcommand.go:21 +0x1cf
github.com/conjurinc/summon/command.runAction(0xc42005a100, 0x0, 0x0, 0x0, 0x0)
/go/src/github.com/conjurinc/summon/command/action.go:130 +0x8ad
github.com/conjurinc/summon/command.glob..func1(0xc4200880e0)
/go/src/github.com/conjurinc/summon/command/action.go:47 +0x3fd
github.com/conjurinc/summon/vendor/github.com/codegangsta/cli.(*App).Run(0xc42008a000, 0xc42000e0b0, 0xb, 0xb, 0x0, 0x0)
/go/src/github.com/conjurinc/summon/vendor/github.com/codegangsta/cli/app.go:159 +0x54a
main.RunCLI(0x0, 0x0)
/go/src/github.com/conjurinc/summon/cli.go:27 +0x149
main.main()
/go/src/github.com/conjurinc/summon/main.go:9 +0x26

TEST_HARNESS is set to conjurinc/summon-acceptance:v1. It's not clear what this dependency is or where it came from.

I have just created simple provider for Keepass kdbx file - mskarbek/summon-keepass.

Summon needs to fail more gracefully and not panic when an unknown flag is passed.

Current behavior:

panic: flag provided but not defined: -e

goroutine 1 [running]:
main.main()
    /go/src/github.com/conjurinc/summon/main.go:5 +0x59

Summon fails to pass signals into the child process, instead, it simply terminates the child process when receiving a kill signal. This means it can't be used in environments that depend on graceful shutdown of an application (like rolling restarts in OpenShift/Kuberneted).

Desired Result : Summon passes the common kill signals to the child process, waiting a reasonable period of time to terminate the process, before killing it.

This issue is to track the enhancement of the secrets.yml spec to allow an optional provider specifier. This will allow you to use different providers to resolve secrets in the same secrets.yml file.

Proposal is to add a provider YML field to specify what provider should resolve a secret. If no provider field exists, the default provider is used.

common should always be inherited but it would be awesome to be able to inherit other sections

common:
  key1:
  key2:

appShare:
  pathA:
  keyB:

appA:
  - inherit appShare
  keyC:

appB:
  - inherit appShare
  keyD:

It would be useful to support default values for substitution variables in secrets.yml using bash syntax:

ENV=${a:-default}

This will allow to have more flexibility without having to specify a lot of parameters that you don't care about.

https://cyberark.github.io/summon/ -- see for example the docker example link.

Also the Examples and the Providers links

(Issue created on behalf of @jeniaSakirko.)

To test install.sh, we run it in Docker containers to test that it works okay. This is fine, but creates a problem for releases. install.sh inspects the local version and tries to pull that from github releases, but when you're releasing a new version that release isn't up yet and the build fails.

Examples

• https://gitlab.com/cyberark/summon/pipelines/30219999
• https://jenkins.conjur.net/job/cyberark--summon/job/master/208/console

We need to rethink how we run this test as part of our release process.

Using this script as /usr/libexec/summon/test.sh:

#!/bin/bash
echo "SECRET: '$1'"

Running summon -p test.sh --yaml 'TEST: !var testing' env does not work.
But running summon --yaml 'TEST: !var testing' env does work.

Need to call fullpath in provider.go line 20

Given the following secrets.yml:

common:
    FOO: foo

production:
    BAR: bar

Here is some observed behavior:

$ summon -e production printenv | egrep 'FOO|BAR'
BAR=bar
FOO=foo
$ summon -e common printenv | egrep 'FOO|BAR'
FOO=foo
$ summon printenv | egrep 'FOO|BAR'
$

The first two observations match my expectations based on the documentation.

The third example's behavior isn't mentioned in the documentation, so I won't go so far as to call it a bug.

However, it is mildly surprising. Since common is always inherited by other sections, I kinda had a weak expectation that its values would always get injected, even if we do not pass -e on the command line.

Docs should be elaborated with examples to describe the behavior in more details between root vars and common section as it changes when using and omitting the -e variable.

When I try to install into a running container (without sudo, the install script fails with the following:

root@0a35bbc43151:/# curl -sSL https://raw.githubusercontent.com/cyberark/summon/master/install.sh | bash
Downloading https://github.com/cyberark/summon/releases/download/v0.6.6/summon-linux-amd64.tar.gz
Installing summon v0.6.6 into /usr/local/bin
bash: line 58: sudo: command not found

One use case I'm solving for involves fetching a secret via Summon's !var:file tag and then symlinking a known path to the temp file stored in that environment variable.

Native support for this kind of symlinking would help me avoid my current undesired solution of running summon -p ... -f ... symlinker.sh myscript.sh

Gopass can be used as provider out of the box. Just added usage as summon provider to the gopass documentation. Would be nice to link here as well:

https://github.com/justwatchcom/gopass/blob/master/docs/summon-provider.md

Under the "Install" section, the "Provider of your choice" URL points to http://conjurinc.github.io/summon/#providers which is currently a broken link.

Unless you read documentation from top to bottom It's not clear that except downloading Summon itself it is necessary to download provider implementation in addition and put into /usr/local/lib/summon/dir. Please add installation notes section and explain it explicitly. Another option would be to provide friendly installer which will do this automatically

It has been suggested that we write a provider for the Chef API to make the transition from secrets stored in Chef easier.

Maybe we can use this lib https://github.com/marpaia/chef-golang. The provider will take care of decryption

If in my secrets.yml I set the value of a key to an integer the key is not exported

example of exported and not exported specifications in secrets.yml

EXPORTED: '3306'
NOT_EXPORTED: 3306

no error is thrown either way.
In the erroneous case the only difference is the secret is not exported

Process a Go template, substituting secrets values into the template.

It would appear our hard coded value is making summon ignore what is provided with the --provider switch.

Summon checks for executable bit when executing a provider and fails if it's not present.

Unfortunately, this doesn't work on Windows which doesn't have executable bits.

it looks like the generated @SUMMONENVFILE does not include a newline character on the last line. This leads to shell utilities adhering to POSIX standards ignoring the last line when reading the file.

Probably the most secure way for one application to give information is through a file which was created, opened, then removed. Here's a suggestion about how it might be implemented on UNIX-like systems:

• Create a tmp file readable only by owner (mode 0600), and opened for read-write
• unlink (remove) the temporary file
• Write the name=value contents into it
• seek to the beginning
• pass the file descriptor number to the client program

The client program then reads the file, gets the secrets it wants, then closes the file. No other process on the system can at any time see the contents unless they open it before its been removed. Of course, someone with access to the block devices under the OS could read it. But then you're screwed anyway ;-).

After #84 I can no longer install summon using install.sh, it also fails installing it on docker container during deployment.
It stops at do_download and returns status code 8.
Do you guys know what's the issue? Is there any easy way to manage installed summon versions in Dockerfile with ubuntu image?

This is a problem when installing providers, unless you disable SSI OSX won't let you place them in /usr/libexec/summon, the default provider directory. We need to change the default provider dir for OSX.

Many software packages read secret from configuration files and would have to be modified to source them from the environment instead. This makes it cumbersome to use them with summon; the usual solution is to write a wrapper script which seds the secrets into place before handing off to target command and use summon to call that script instead.

Not only is this inconvenient, but also risks introducing security problems: the script author needs to remember to clean up the files afterwards, give them correct permissions to limit exposure, etc. This would much better be handled by summon itself, at the small cost of not being agnostic about templating engine.

I propose thus to design and implement functionality in summon that would allow using file templates into which the secrets would get substituted. The templates would be rendered into temporary files as with !file entries currently, to be cleaned up after exiting; additionally these temp files could be symlinked into a required place before calling the target process so that the target can find them (on the assumption that it cannot use environment variables for that), or alternatively we could provide a mechanism to substitute the temp file path into the command line.

Note this functionality would replace the obsolete conjur env command of https://github.com/cyberark/conjur-cli.

Install script throws on curl, looks like misplaced flag

root@3621d7cc3971:/# curl -sSL https://raw.githubusercontent.com/cyberark/summon/master/install.sh | bash
curl: option -C: expected a positive numerical parameter
curl: try 'curl --help' or 'curl --manual' for more information
cat: /tmp/install.sh.3281/summon.version: No such file or directory
Downloading https://github.com/cyberark/summon/releases/download/v/summon-linux-amd64.tar.gz
curl: option -C: expected a positive numerical parameter
curl: try 'curl --help' or 'curl --manual' for more information

Here's a workaround

install=$(curl -sSL https://raw.githubusercontent.com/cyberark/summon/master/install.sh); echo "${install/curl -sSL -C/curl -sSL}" | bash

Hi

Possibly a dumb question, but how should the summon-s3 provider be installed? Is it clone from git, build, package and then move the dist package to /usr/libexec/summon/?

Thanks
Rob

Are there any significant roadblocks that prevent "LastPass" integration?

When attempting to create a docker image for summon on alpine linux, I am unable to get summon to retrieve secrets. I have only tried the s3 provider. It works just fine if I use ubuntu as a base image rather than Alpine. I'm at a loss and not sure how to debug further. Any ideas? Below is the sh trace

/ # summon --yaml 'test: !var myorg-creds/test' cat @SUMMONENVFILE
Error fetching variable test: 

repo is missing its license

Summon should be able to pass signals to the subprocess properly. We can do this by using syscall.Exec.

AC:

• Use syscall.Exec to launch subprocesses
• (Optional) Cleanup the codebase a bit

Because summon's temp file is under /var I can't mount it as a Docker volume under OSX with VirtualBox.

For example, the Docker CLI accepts an --env-file argument, which is a file containing environment variable names and values.

https://docs.docker.com/reference/commandline/cli/

Cauldron could be instructed to insert an --env-file argument into the sub-command, the value of which would be a temp file containing the secrets in env file format.

Hello,

We are interested in using summon for secret management but we use a pre forking server like Unicorn. This means that the forked process wont have the ENV variables,

How does this work with summon? Couldn't find any documentation on it

Regards,

summon build occasionally failing in jenkins

It's generally easy and straightforward and will future-proof the project

AC:

• Project is built using go modules
• Documentation is updated to reflect this

As an automated system, I want a minimal Summon install script without any providers so that I can install only the providers I want, using my own tools, into my preferred location.

As a human, I want a single install script that includes with Summon a few common useful providers in a sensible default location.

Right now, the Summon install experience is catered to the first (automation) persona, and humans have to go through multiple manual steps to install Summon + providers.

Summon should rename the current minimal install script to install-minimal.sh and create a new install.sh script which includes some providers, so that after one step a human user is ready to ~ begin ~ summoning ~

Acceptance criteria:

GIVEN I am looking at the Summon readme,
WHEN I follow the "install" instructions,
THEN I get Summon + providers installed into the default location so that I can immediately fetch secrets;
WHEN I follow the "minimal install (expert)" directions,
THEN I get Summon alone

Feature request that if the -e foo switch is used, that the value foo be injected into the subprocess as well as a new variable - perhaps it would be something like SUMMON_ENV or similar.

The use case here is for the subprocess to exhibit different behavior, depending on which environment it detects while preventing the need to keep redefining the same variable in multiple sections.

Expected behavior:

$ summon -e foo env
...
SUMMON_ENV=foo
...

$ summon -e bar env
...
SUMMON_ENV=bar
...

$ summon env
...
<No extra variables>
...

Thanks for writing summon, we've found it very useful!

At present, summon --version only gives you part of the story, because it doesn't include the versions of known providers. It could be more useful if those versions were also printed.

The question of how to identify all installed providers is an interesting one, because any program or script could perhaps be a Summon provider. I propose that Summon provider installers ought to register themselves somehow, and that using Summon with any script as a provider should register that script.

When you run summon --version it can run $provider --version for each provider, and on a zero exit status it will include the provider's version alongside Summon's.

It would be more standard to build with a Makefile rather than build.sh

In the providers section, it says the env var is summon_PROVIDER, it should be SUMMON_PROVIDER.

IMO https://www.vaultproject.io/ is perfect match for this tool.

Launching a subprocess with summon that has flags that summon also has causes summon to fail.

Example:

summon psql -p 7432 -h conjur -U postgres

summon and psql both use the -p flag. summon doesn't know this is a flag for the psql process. This needs to be fixed.

My SUMMONENVFILE can't be interpreted by Docker.

✗ summon docker run --env-file "@SUMMONENVFILE" myapp
docker: poorly formatted environment: variable '-----END RSA PRIVATE KEY-----' has white spaces.

Here's what it looks like:

✗ summon cat @SUMMONENVFILE                                        
SSH_PRIVATE_KEY_PATH=-----BEGIN RSA PRIVATE KEY-----
MIIEpAIBAAKCAQEAqLVITDJKy1KHfHG/SKvM07CjP/D7XrAr3Mz2UKzslnxgJKgeA8hENPGuj83Q
...
iY5dzCLuKMiIiPUHfDtrqaNLH+anyJtjmW9Fb1cWFOJLhWVkJAvYeVawwFBooUK1h9cPR728ngsa
5F7/sDUzQZBmRYnpwWFrWrEE1NwvsIWsQ31WIRhs/Xxp9A36ZPnxXWyOtw0XTbd8PmBaYA==
-----END RSA PRIVATE KEY-----

We recently discovered that summon memory usage can increase significantly over the course of its lifetime. By looking at the code, it seems that it is tee-ing stdout to an internal buffer. For long running processes that prints to stdout (like a server using a console appender for example) or any process that writes a lot to stdout, this can lead to very high memory consumption by summon and ultimately, the death of the process.

To reproduce, use a script like

#!/bin/sh

while true; do cat SOME_BIG_FILE; done

Then run this script with summon.

Found in summon version 0.6.5