Reference secret values from Kubernetes secrets about woodpecker HOT 10 OPEN

Yes, that makes sense as well. We could have an implementation like this:

steps:
  - name: test
    image: alpine
    commands:
      - echo $my_password
    backend_options:
      kubernetes:
        secrets:
          - name: my_password
            secretRef: my-top-secret-password
            key: password

from woodpecker.

Thanks for the explanation. That makes a lot of sense. Keeping it with the other Kubernetes backend options seems like it would definitely be the path of least resistance.

Yes, we could support annotations like this maybe:

apiVersion: v1
kind: Secret
type: Opaque
metadata:
  name: my-pipeline-secret
  annotations:
    woodpecker-ci.org/org-access: "my-org,another-org"
    woodpecker-ci.org/repo-access: "my-repo,another-one"
data:
  password: d29vZHBlY2tlciByb2NrcyE=

from woodpecker.

An alternative would be to have all implementation specific parts in 'backend_options' - 'kubernetes'

from woodpecker.

Having last updates, syntax below came up in my mind.

apiVersion: v1
kind: Secret
metadata:
  name: aws
  namespace: woodpecker
data:
  key-id: N0lrZHNzb0xleGFtcGxlNkpJVnZDRXE=
  access-key: TXk0WE5BYXNleGFtcGxldXRKRHNUWHc=
type: Opaque

1. Map an entire secret to the environment:

backend_options:
  kubernetes:
    secrets:
      - name: aws

2. Select a key:

backend_options:
  kubernetes:
    secrets:
      - name: aws
        key: access-key

3. Map to an alternate name:

backend_options:
  kubernetes:
    secrets:
      - name: aws
        key: key-id
        target:
          env: AWS_ACCESS_KEY_ID
      - name: aws
        key: access-key
        target:
          env: AWS_SECRET_ACCESS_KEY

4. Mount as a file:

apiVersion: v1
type: kubernetes.io/dockerconfigjson
kind: Secret
metadata:
  name: reg-cred
  namespace: woodpcker
data:
  .dockerconfigjson: <base64 encoded JSON auth file here>

backend_options:
  kubernetes:
    secrets:
      - name: reg-cred
        key: ".dockerconfigjson"
        target:
          file: "~/.docker/config.json"

What do you think?

from woodpecker.

As I understand FR:
As Admin I want to let users to use (refer to) predefined (existing) Kubernetes secrets in their pipelines.
Am I right?

There might be security concerns... However, if we run workload in separate namespace, then probably OK. Feature can be gated by Agent's flag.

store the information in global env variables, but that doesn't seem ideal from a security standpoint

There are global and org secrets.

I can't push them to my repo when I'm done.

You can use WP secrets :)

allow for other external secret providers as well (see #929)

This is another thing. At least from implementation point of view.

from woodpecker.

Yes, that's the general idea, and yes, security is definitely something to think through here. I see a couple of options off the top of my head:

1. We could require that Secrets have a special annotation to allows them to be accessed by WP pipelines.
2. We could whitelist secret names in the WP server or agent config.

I like the annotation option because is provides for some flexibility. Depending on how the annotation is structured, it could even limit the scope of access to the Secret to a specific org or even repo.

You can use WP secrets :) There are global and org secrets.

Yes, I'm aware of WP secrets at the global and org level. But, since they have to be defined manually, keeping them in-sync with the Kubernetes Secrets becomes a problem.

This is another thing. At least from implementation point of view.

I'm not sure why integrating with other external secret providers has be different from an implementation point of view. But, I'm not very familiar with the WP codebase.

from woodpecker.

I'm not sure why integrating with other external secret providers has be different from an implementation point of view. But, I'm not very familiar with the WP codebase.

Because it is server-side. Server parses pipeline, goes to external provider, loads secret, then set env vars for Agent/backend.

And your example is purely Agent/backend side. I can just translate it into secretKeyRef Pod's definition.

1. Like repo name?
   Where it should be processed?
   If in Server, then server should be able to communicate with Kubernetes. More dependencies and configs (which already in Agent).
   If in Agent, when logic goes to... Agent :) Agent loads secret and checks if there is annotation with current repo name. Seems, not so difficult, but not just translation either.

2. This is awkward to me. Like hardcoded privileged plugins. Besides, you should store it somewhere (CRUD). Perhaps in DB => new Server GUI, etc. Too difficult, I think.

or agent config

It's just dirty hack :)

from woodpecker.

To be done:

• Map an entire secret to environment (Configure all key-value pairs in a Secret as container environment variables, #3655)
• Allow to select keys and map them to alternate names (Define container environment variables using Secret data)
• Mount .dockerconfigjson secrets to a file (Create a Pod that has access to the secret data through a Volume, #3655 (comment))
• Implement access controls (#3582 (comment))
• Implement secret masking (#3655 (comment))
• Write documentation

from woodpecker.

I love it. This syntax seems perfect to me.

from woodpecker.

Being able to do the same things with ConfigMap's would be super swell for things like certificate bundles and site-specific config files that don't belong in git, and the logic should be nearly copy-paste.

from woodpecker.