Injector with External vault service about vault-k8s HOT 11 CLOSED

Hi @whume, I believe you're using the wrong service account. Vault requires RBAC to communicate with Kubernetes to verify service accounts. Looks like you're using the injector service account which is not the same thing.

Create the service account:

---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: vault-auth
  namespace: vault
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  name: role-tokenreview-binding
  namespace: vault
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: system:auth-delegator
subjects:
  - kind: ServiceAccount
    name: vault-auth
    namespace: vault

Get the JWT for that service account:

secret_name="$(kubectl get serviceaccount vault-auth -n vault -o go-template='{{ (index .secrets 0).name }}')"
tr_account_token="$(kubectl get secret ${secret_name} -n vault -o go-template='{{ .data.token }}' | base64 --decode)"

Setup the Vault auth method using that new JWT:

vault write auth/cluster-01/config token_reviewer_jwt="${tr_account_token}" kubernetes_host="${k8s_host}" kubernetes_ca_cert="${k8s_cacert}"

from vault-k8s.

Hi @whume ,

Token rotations are based on the token_max_ttl for the role:

vault write auth/kubernetes/role/db-app \
    bound_service_account_names=app \
    bound_service_account_namespaces=app \
    policies=app \
    token_max_ttl=60s \
    ttl=30s

Secret rotations are based on the secrets TTL.

Hope that helps! If you're satisfied with these answers, please close the issue!

from vault-k8s.

@chinmaychandak @pksurferdad - How did you guys resolved the issue. My am facing the same issue with 2 private GKE cluster.
Error in Vault cluster logs -
[ERROR] auth.kubernetes.auth_kubernetes_08c36485: login unauthorized due to: Post "https://<app_ip_cluster>/apis/authentication.k8s.io/v1/tokenreviews": dial tcp <app_ip_cluster:443: i/o timeout

and application cluster
"errors": [ 46 "permission denied"
I am using following link - (external vault ) https://learn.hashicorp.com/tutorials/vault/agent-kubernetes

from vault-k8s.

Sadly I am still getting the same thing after applying the above. Error has not changed just permission denied

from vault-k8s.

So we figured it out. Turns out everything in the cluster was good Vault was being blocked by the firewall trying to send back to the cluster. We are good as far as the issue. If you don't mind I have one more question. How is the TTL for the rotation on the secret handled. Is that controlled by the TTL vault role or is there a setting to adjust that?

Thanks for the help.

from vault-k8s.

Perfect Thank you

from vault-k8s.

Hi @whume I know this issue is closed but do you remember what the firewall issue was here #69 (comment)? Did you have top allow 8080 TCP traffic? I'm running vault on a stand alone AWS eks k8s cluster and I'm trying to connect to the vault server from another eks cluster .

from vault-k8s.

@whume, @pksurferdad I'm facing the exact same issue - how were you guys able to solve the firewall issue? I'm using the Vault Agent Injector on an AWS EKS standalone K8s cluster against an external Vault.

from vault-k8s.

sorry @chinmaychandak, is issue turned out not to be firewall related, but once i had the kubernetes auth config correct in vault, my connections were fine; however, i'm running everything on k8s, but needed a k8s cluster to talk to vault on another k8s cluster.

from vault-k8s.

@pksurferdad, thanks for responding! Yes, I got my setup working too - same issue where I had to reconfigure K8s auth in Vault

from vault-k8s.

@asl-cloud99, I was able to fix my issues by ensuring 2 things:

• That the K8s Client Cluster's auth config was correctly set up in Vault (try reconfiguring this from scratch; in my case, the K8s JWT_TOKEN had changed because of some changes, and so reconfiguring the auth in Vault helped)
• See if your Client and Vault can talk to each other, i.e. I modified their network inbound rules and such on AWS. You might need to do the same on GCP.

from vault-k8s.