Code Monkey home page Code Monkey logo

Comments (3)

bergerx avatar bergerx commented on August 11, 2024

@luisdavim did you check the deployment yaml files i linked below.

According to https://github.com/kubernetes/community/blob/master/contributors/devel/flexvolume.md, vendor drivers should be installed in the volume plugin path on every node, for Azure specific drivers, it seems like they can be installed in a DeamonSet form with using hostPath mount points. e.g.:

With one caution, apparently these deployment files are designed for AKS or acs-engine/aks-engine which both uses --volume-plugin-dir=/etc/kubernetes/volumeplugins for kubelets. But the default value of the --volume-plugin-dir parameter is /usr/libexec/kubernetes/kubelet-plugins/volume/exec/. Seems like deployment yaml files are mostly for AKS/aks-engine and may need some tuning when using with Gardener.

Also i'm not sure if we can use hostPath's in our shoots by default, last time i checked PSP's they werre not prevented but thats something to consider.

from gardener-extension-provider-azure.

luisdavim avatar luisdavim commented on August 11, 2024

I've tried the DaemonSet approach but it doesn't seem to work:

$ cat blobfuse-flexvol-installer-1.9.yaml
apiVersion: v1
kind: Namespace
metadata:
  name: flex
  labels:
    type: system
---
apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: blobfuse-flexvol-installer
  namespace: flex
  labels:
    k8s-app: blobfuse
spec:
  selector:
    matchLabels:
      name: blobfuse
  template:
    metadata:
      labels:
        name: blobfuse
    spec:
      containers:
      - name: blobfuse-flexvol-installer
        image: mcr.microsoft.com/k8s/flexvolume/blobfuse-flexvolume
        imagePullPolicy: Always
        volumeMounts:
        - name: volplugins
          mountPath: /usr/libexec/kubernetes/kubelet-plugins/volume/exec/
        - name: varlog
          mountPath: /var/log/
      volumes:
      - name: varlog
        hostPath:
          path: /var/log/
      - name: volplugins
        hostPath:
          path: /usr/libexec/kubernetes/kubelet-plugins/volume/exec/
$ kctl get ds -n flex
NAME                         DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR   AGE
blobfuse-flexvol-installer   4         4         0       4            0           <none>          73s
$ kctl describe po blobfuse-flexvol-installer-4ctr7 -n flex
Name:               blobfuse-flexvol-installer-4ctr7
Namespace:          flex
Priority:           0
PriorityClassName:  <none>
Node:               shoot--studio--studio-cpu-worker-66bd458847-vtd6t/10.250.0.9
Start Time:         Thu, 03 Jan 2019 09:32:38 +0000
Labels:             controller-revision-hash=5d9655ccf7
                    name=blobfuse
                    pod-template-generation=1
Annotations:        cni.projectcalico.org/podIP: 100.96.3.125/32
                    kubernetes.io/psp: gardener.privileged
Status:             Running
IP:                 100.96.3.125
Controlled By:      DaemonSet/blobfuse-flexvol-installer
Containers:
  blobfuse-flexvol-installer:
    Container ID:   docker://cd394c1535752172061037fb296d51ba10a7b394b1e50db3d30cd6bfcb472566
    Image:          mcr.microsoft.com/k8s/flexvolume/blobfuse-flexvolume
    Image ID:       docker-pullable://mcr.microsoft.com/k8s/flexvolume/blobfuse-flexvolume@sha256:f9822fd37987419f28c6fa26e3e9838bdd5f46bf8acf3a23cf471aa672607005
    Port:           <none>
    Host Port:      <none>
    State:          Waiting
      Reason:       CrashLoopBackOff
    Last State:     Terminated
      Reason:       ContainerCannotRun
      Message:      error while creating mount source path '/usr/libexec/kubernetes/kubelet-plugins/volume/exec': mkdir /usr/libexec/kubernetes: read-only file system
      Exit Code:    128
      Started:      Thu, 03 Jan 2019 09:34:00 +0000
      Finished:     Thu, 03 Jan 2019 09:34:00 +0000
    Ready:          False
    Restart Count:  4
    Environment:    <none>
    Mounts:
      /usr/libexec/kubernetes/kubelet-plugins/volume/exec/ from volplugins (rw)
      /var/log/ from varlog (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-ctq79 (ro)
Conditions:
  Type              Status
  Initialized       True
  Ready             False
  ContainersReady   False
  PodScheduled      True
Volumes:
  varlog:
    Type:          HostPath (bare host directory volume)
    Path:          /var/log/
    HostPathType:
  volplugins:
    Type:          HostPath (bare host directory volume)
    Path:          /usr/libexec/kubernetes/kubelet-plugins/volume/exec/
    HostPathType:
  default-token-ctq79:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  default-token-ctq79
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/disk-pressure:NoSchedule
                 node.kubernetes.io/memory-pressure:NoSchedule
                 node.kubernetes.io/not-ready:NoExecute
                 node.kubernetes.io/unreachable:NoExecute
                 node.kubernetes.io/unschedulable:NoSchedule
Events:
  Type     Reason     Age                  From                                                        Message
  ----     ------     ----                 ----                                                        -------
  Normal   Scheduled  3m5s                 default-scheduler                                           Successfully assigned flex/blobfuse-flexvol-installer-4ctr7 to shoot--studio--studio-cpu-worker-66bd458847-vtd6t
  Normal   Pulling    103s (x5 over 3m4s)  kubelet, shoot--studio--studio-cpu-worker-66bd458847-vtd6t  pulling image "mcr.microsoft.com/k8s/flexvolume/blobfuse-flexvolume"
  Normal   Pulled     103s (x5 over 3m2s)  kubelet, shoot--studio--studio-cpu-worker-66bd458847-vtd6t  Successfully pulled image "mcr.microsoft.com/k8s/flexvolume/blobfuse-flexvolume"
  Normal   Created    102s (x5 over 3m2s)  kubelet, shoot--studio--studio-cpu-worker-66bd458847-vtd6t  Created container
  Warning  Failed     102s (x5 over 3m2s)  kubelet, shoot--studio--studio-cpu-worker-66bd458847-vtd6t  Error: failed to start container "blobfuse-flexvol-installer": Error response from daemon: error while creating mount source path '/usr/libexec/kubernetes/kubelet-plugins/volume/exec': mkdir /usr/libexec/kubernetes: read-only file system
  Warning  BackOff    89s (x5 over 2m34s)  kubelet, shoot--studio--studio-cpu-worker-66bd458847-vtd6t  Back-off restarting failed container

from gardener-extension-provider-azure.

rfranzke avatar rfranzke commented on August 11, 2024

I don't think we will do that soon @luisdavim, maybe we can address this one after extensibility? However, if you have capacity, feel free to file a PR if that might be too long for you.

from gardener-extension-provider-azure.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.