Code Monkey home page Code Monkey logo

Comments (15)

FrimIdan avatar FrimIdan commented on August 17, 2024 2

Hi @mtcolman found the following commit in go-containerregistry google/go-containerregistry@892d7a8 pushed 14 days ago.
It should fix that issue here, we will update go.mod and will add a test case.

from kubeclarity.

mtcolman avatar mtcolman commented on August 17, 2024 1

@FrimIdan tested and looks good, yes I believe you can close. Thanks for speedy fix.

from kubeclarity.

mmurhamm avatar mmurhamm commented on August 17, 2024

The expected behavior described above actually worked with kubei (1.0.15); it would be helpful to have somewhat more detailed documentation about how kubeclarity works internally and how it differs from kubei. Thanks.

from kubeclarity.

mtcolman avatar mtcolman commented on August 17, 2024

I increased the log-level and have also now seen this:

{IMAGE_NAME_TO_SCAN uk.icr.io/sbu-pipeline/zap2docker-stable:2.11.1-sbu nil} {K8S_IMAGE_PULL_SECRET  &EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:&SecretKeySelector{LocalObjectReference:LocalObjectReference{Name:basic-regcred,},Key:.dockerconfigjson,Optional:nil,},}}]

and this:

time="2022-04-26T18:19:50Z" level=debug msg="No matching secret found." func=github.com/cisco-open/kubei/shared/pkg/utils/k8s.GetMatchingSecretName file="/build/shared/pkg/utils/k8s/k8s_utils.go:71"

Which I believe relates to lines 202 and 231 of this code whereby the image pull secret is supposedly obtained and then the matching secret obtained as well.

from kubeclarity.

akpsgit avatar akpsgit commented on August 17, 2024

Hi @mtcolman @mmurhamm, thanks for the information. We'll add documents about the architectural changes between Kubei 1 and a Kubei 2 to make it more clear. Regarding the image pull secret, the main difference is that we moved from using DockerKeyRing https://github.com/cisco-open/kubei/blob/1.0.16/pkg/utils/k8s/k8s_utils.go#L57 to use
"github.com/google/go-containerregistry/pkg/authn/kubernetes:
https://github.com/cisco-open/kubei/blob/3217eb2e7da5b7d05952dc9711cc1ad9efd2b0cb/shared/pkg/utils/k8s/k8s_utils.go#L75

From the provided logs, looks like the basic-regcred secret should be used (no ImagePullSecret on the pod), right?

from kubeclarity.

mtcolman avatar mtcolman commented on August 17, 2024

@akpsgit thanks for your reply. The imagePullSecret is in the pod to be scanned, so it would expect it to use that rather than falling back to basic-regcred:

$ oc get pod matt-zap-owaspzap-zap-66cb56cdc4-wkxlw -n sbu-dev -oyaml | grep PullSecrets -A2
  imagePullSecrets:
  - name: sbu-pipeline

Secret is present in that namespace as well:

$ oc get secret -n sbu-dev | grep sbu
sbu-pipeline                                         kubernetes.io/dockerconfigjson        1      152d

from kubeclarity.

akpsgit avatar akpsgit commented on August 17, 2024

Is matt-zap-owaspzap-zap-66cb56cdc4-wkxlw is the only pod that contains the uk.icr.io/sbu-pipeline/zap2docker-stable:2.11 image? I'm asking because we create a scanner per image, and take the details from the first pod in the list that contains this image (might not have the pull secret) - small chance but worth checking:
https://github.com/cisco-open/kubei/blob/3217eb2e7da5b7d05952dc9711cc1ad9efd2b0cb/runtime_scan/pkg/scanner/job_managment.go#L262

If there is no other pod that uses this image, worth trying to start this pod in a separate namespace and scan only it to get more coherent logs in KubeClarity backend (where the job management is running). If nothing new comes up, can you please attach the "oc get pod matt-zap-owaspzap-zap-66cb56cdc4-wkxlw -n sbu-dev -oyaml" output (with stripped sensitive/identifying values) so we can try to recreate and debug it?

from kubeclarity.

mtcolman avatar mtcolman commented on August 17, 2024

I can confirm it is the only pod that uses this image:

$ oc get pod -n sbu-dev -oyaml | grep zap2docker -A5 -B5
      - name: ZAP_APIKEY
        valueFrom:
          secretKeyRef:
            key: zap-api-key
            name: matt-zap-zap-key
      image: uk.icr.io/sbu-pipeline/zap2docker-stable:2.11.1-sbu
      imagePullPolicy: IfNotPresent
      name: owaspzap
      ports:
      - containerPort: 8081
        protocol: TCP
--
      lastTransitionTime: "2022-03-31T03:09:26Z"
      status: "True"
      type: PodScheduled
    containerStatuses:
    - containerID: cri-o://d15f80077fc96558e7a0e36202d8b55e4051f21d53668abac9614b6ada558707
      image: uk.icr.io/sbu-pipeline/zap2docker-stable:2.11.1-sbu
      imageID: uk.icr.io/sbu-pipeline/zap2docker-stable@sha256:6c9d3f2cc80470bb4b54fb4b402ff982905e5cb2f13648b571da37e277540f00
      lastState:
        terminated:
          containerID: cri-o://4897b696c88bcb0bf8d067d83af59001c630e481b230c569107f6bb3e44919c7
          exitCode: 137
          finishedAt: "2022-04-27T14:18:37Z"

from kubeclarity.

mtcolman avatar mtcolman commented on August 17, 2024

Created brand new namespace mattzap and deployed pod. That via UI, selected namespace and ran scan:

$ oc get pod -n mattzap
NAME                                                              READY   STATUS      RESTARTS   AGE
newzap-owaspzap-zap-b9dc5f9d7-qbm44                               2/2     Running     0          2m32s
scanner-nginx-unprivileged-e34e22b6-7eab-4d4a-a45a-fc3--1-sg4bw   0/1     Completed   0          54s
scanner-zap2docker-stable-78eba9a6-6014-49bb-8331-2dac--1-pvndz   0/1     Completed   0          54s

Scan results are:

time="2022-04-27T16:08:52Z" level=info msg="Start scanning..." func="github.com/cisco-open/kubei/runtime_scan/pkg/scanner.(*Scanner).Scan" file="/build/runtime_scan/pkg/scanner/scanner.go:265" scanner id=f880455f-e909-4fa5-8d7f-2c38a92d66c2
time="2022-04-27T16:08:52Z" level=debug msg="No matching secret found." func=github.com/cisco-open/kubei/shared/pkg/utils/k8s.GetMatchingSecretName file="/build/shared/pkg/utils/k8s/k8s_utils.go:71"
time="2022-04-27T16:08:52Z" level=debug msg="No matching secret found." func=github.com/cisco-open/kubei/shared/pkg/utils/k8s.GetMatchingSecretName file="/build/shared/pkg/utils/k8s/k8s_utils.go:71"
time="2022-04-27T16:08:52Z" level=info msg="Total 2 unique images to scan" func="github.com/cisco-open/kubei/runtime_scan/pkg/scanner.(*Scanner).initScan" file="/build/runtime_scan/pkg/scanner/scanner.go:255" scanner id=f880455f-e909-4fa5-8d7f-2c38a92d66c2
time="2022-04-27T16:08:52Z" level=debug msg="Created job=&Job{ObjectMeta:{scanner-nginx-unprivileged-e34e22b6-7eab-4d4a-a45a-fc3366394ca3  mattzap    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[app:scanner sidecar.istio.io/inject:false] map[] [] []  []},Spec:JobSpec{Parallelism:nil,Completions:nil,ActiveDeadlineSeconds:nil,Selector:nil,ManualSelector:nil,Template:{{      0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[app:scanner sidecar.istio.io/inject:false] map[] [] []  []} {[{tmp-volume {nil EmptyDirVolumeSource{Medium:,SizeLimit:<nil>,} nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil}}] [] [{vulnerability-scanner ghcr.io/cisco-open/kubeclarity-runtime-k8s-scanner:latest [] [scan --log-level debug]  [] [] [{RESULT_SERVICE_ADDR kubeclarity-kubeclarity.kubeclarity:8888 nil} {SBOM_DB_ADDR kubeclarity-kubeclarity-sbom-db.kubeclarity:8080 nil} {ANALYZER_LIST syft gomod nil} {ANALYZER_SCOPE squashed nil} {SCANNERS_LIST grype nil} {SCANNER_GRYPE_MODE REMOTE nil} {REGISTRY_SKIP_VERIFY_TLS false nil} {REGISTRY_USE_HTTP false nil} {SCANNER_REMOTE_GRYPE_SERVER_ADDRESS kubeclarity-kubeclarity-grype-server.kubeclarity:9991 nil} {SCANNER_REMOTE_GRYPE_SERVER_TIMEOUT 2m nil} {SCAN_UUID 4a32c0a3-296b-440c-839c-a0f1478e1098 nil} {IMAGE_ID_TO_SCAN uk.icr.io/sbu-pipeline/nginx-unprivileged@sha256:e86606220600fb7665e74453ca4e4b46a03215ea312e621c02809d2242a4bea2 nil} {IMAGE_HASH_TO_SCAN e86606220600fb7665e74453ca4e4b46a03215ea312e621c02809d2242a4bea2 nil} {IMAGE_NAME_TO_SCAN uk.icr.io/sbu-pipeline/nginx-unprivileged:1.21.4-alpine-sbu nil}] {map[cpu:{{1000 -3} {<nil>}  DecimalSI} memory:{{1048576000 0} {<nil>}  BinarySI}] map[cpu:{{50 -3} {<nil>} 50m DecimalSI} memory:{{52428800 0} {<nil>} 50Mi BinarySI}]} [{tmp-volume false /tmp  <nil> }] [] nil nil nil nil   Always &SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[all],},Privileged:*false,SELinuxOptions:nil,RunAsUser:nil,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,} false false false}] [] Never <nil> <nil>  map[]   <nil>  false false false <nil> nil []   nil  [] []  <nil> nil [] <nil> <nil> <nil> map[] [] <nil> nil}},BackoffLimit:*0,TTLSecondsAfterFinished:*300,CompletionMode:nil,Suspend:nil,},Status:JobStatus{Conditions:[]JobCondition{},StartTime:<nil>,CompletionTime:<nil>,Active:0,Succeeded:0,Failed:0,CompletedIndexes:,UncountedTerminatedPods:nil,Ready:nil,},}" func="github.com/cisco-open/kubei/runtime_scan/pkg/scanner.(*Scanner).runJob" file="/build/runtime_scan/pkg/scanner/job_managment.go:164" scanner id=f880455f-e909-4fa5-8d7f-2c38a92d66c2
time="2022-04-27T16:08:52Z" level=info msg="Running job mattzap/scanner-nginx-unprivileged-e34e22b6-7eab-4d4a-a45a-fc3366394ca3 to scan imageID uk.icr.io/sbu-pipeline/nginx-unprivileged@sha256:e86606220600fb7665e74453ca4e4b46a03215ea312e621c02809d2242a4bea2" func="github.com/cisco-open/kubei/runtime_scan/pkg/scanner.(*Scanner).runJob" file="/build/runtime_scan/pkg/scanner/job_managment.go:166" scanner id=f880455f-e909-4fa5-8d7f-2c38a92d66c2
time="2022-04-27T16:08:52Z" level=debug msg="Created job=&Job{ObjectMeta:{scanner-zap2docker-stable-78eba9a6-6014-49bb-8331-2daca6ce9373  mattzap    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[app:scanner sidecar.istio.io/inject:false] map[] [] []  []},Spec:JobSpec{Parallelism:nil,Completions:nil,ActiveDeadlineSeconds:nil,Selector:nil,ManualSelector:nil,Template:{{      0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[app:scanner sidecar.istio.io/inject:false] map[] [] []  []} {[{tmp-volume {nil EmptyDirVolumeSource{Medium:,SizeLimit:<nil>,} nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil}}] [] [{vulnerability-scanner ghcr.io/cisco-open/kubeclarity-runtime-k8s-scanner:latest [] [scan --log-level debug]  [] [] [{RESULT_SERVICE_ADDR kubeclarity-kubeclarity.kubeclarity:8888 nil} {SBOM_DB_ADDR kubeclarity-kubeclarity-sbom-db.kubeclarity:8080 nil} {ANALYZER_LIST syft gomod nil} {ANALYZER_SCOPE squashed nil} {SCANNERS_LIST grype nil} {SCANNER_GRYPE_MODE REMOTE nil} {REGISTRY_SKIP_VERIFY_TLS false nil} {REGISTRY_USE_HTTP false nil} {SCANNER_REMOTE_GRYPE_SERVER_ADDRESS kubeclarity-kubeclarity-grype-server.kubeclarity:9991 nil} {SCANNER_REMOTE_GRYPE_SERVER_TIMEOUT 2m nil} {SCAN_UUID c8321846-3f3f-44ea-8bf7-e6ebdb50deb1 nil} {IMAGE_ID_TO_SCAN uk.icr.io/sbu-pipeline/zap2docker-stable@sha256:6c9d3f2cc80470bb4b54fb4b402ff982905e5cb2f13648b571da37e277540f00 nil} {IMAGE_HASH_TO_SCAN 6c9d3f2cc80470bb4b54fb4b402ff982905e5cb2f13648b571da37e277540f00 nil} {IMAGE_NAME_TO_SCAN uk.icr.io/sbu-pipeline/zap2docker-stable:2.11.1-sbu nil}] {map[cpu:{{1000 -3} {<nil>}  DecimalSI} memory:{{1048576000 0} {<nil>}  BinarySI}] map[cpu:{{50 -3} {<nil>} 50m DecimalSI} memory:{{52428800 0} {<nil>} 50Mi BinarySI}]} [{tmp-volume false /tmp  <nil> }] [] nil nil nil nil   Always &SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[all],},Privileged:*false,SELinuxOptions:nil,RunAsUser:nil,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,} false false false}] [] Never <nil> <nil>  map[]   <nil>  false false false <nil> nil []   nil  [] []  <nil> nil [] <nil> <nil> <nil> map[] [] <nil> nil}},BackoffLimit:*0,TTLSecondsAfterFinished:*300,CompletionMode:nil,Suspend:nil,},Status:JobStatus{Conditions:[]JobCondition{},StartTime:<nil>,CompletionTime:<nil>,Active:0,Succeeded:0,Failed:0,CompletedIndexes:,UncountedTerminatedPods:nil,Ready:nil,},}" func="github.com/cisco-open/kubei/runtime_scan/pkg/scanner.(*Scanner).runJob" file="/build/runtime_scan/pkg/scanner/job_managment.go:164" scanner id=f880455f-e909-4fa5-8d7f-2c38a92d66c2
time="2022-04-27T16:08:52Z" level=info msg="Running job mattzap/scanner-zap2docker-stable-78eba9a6-6014-49bb-8331-2daca6ce9373 to scan imageID uk.icr.io/sbu-pipeline/zap2docker-stable@sha256:6c9d3f2cc80470bb4b54fb4b402ff982905e5cb2f13648b571da37e277540f00" func="github.com/cisco-open/kubei/runtime_scan/pkg/scanner.(*Scanner).runJob" file="/build/runtime_scan/pkg/scanner/job_managment.go:166" scanner id=f880455f-e909-4fa5-8d7f-2c38a92d66c2
time="2022-04-27T16:08:53Z" level=info msg="Waiting for result. imageID=uk.icr.io/sbu-pipeline/nginx-unprivileged@sha256:e86606220600fb7665e74453ca4e4b46a03215ea312e621c02809d2242a4bea2" func="github.com/cisco-open/kubei/runtime_scan/pkg/scanner.(*Scanner).waitForResult" file="/build/runtime_scan/pkg/scanner/job_managment.go:135" scanner id=f880455f-e909-4fa5-8d7f-2c38a92d66c2
time="2022-04-27T16:08:53Z" level=info msg="Waiting for result. imageID=uk.icr.io/sbu-pipeline/zap2docker-stable@sha256:6c9d3f2cc80470bb4b54fb4b402ff982905e5cb2f13648b571da37e277540f00" func="github.com/cisco-open/kubei/runtime_scan/pkg/scanner.(*Scanner).waitForResult" file="/build/runtime_scan/pkg/scanner/job_managment.go:135" scanner id=f880455f-e909-4fa5-8d7f-2c38a92d66c2
time="2022-04-27T16:09:01Z" level=info msg="Vulnerabilities result was set for imageID \"uk.icr.io/sbu-pipeline/zap2docker-stable@sha256:6c9d3f2cc80470bb4b54fb4b402ff982905e5cb2f13648b571da37e277540f00\"" func="github.com/cisco-open/kubei/runtime_scan/pkg/scanner.(*Scanner).HandleScanResults" file="/build/runtime_scan/pkg/scanner/scanner.go:392" scanner id=f880455f-e909-4fa5-8d7f-2c38a92d66c2
time="2022-04-27T16:09:01Z" level=warning msg="Vulnerabilities scan of imageID \"uk.icr.io/sbu-pipeline/zap2docker-stable@sha256:6c9d3f2cc80470bb4b54fb4b402ff982905e5cb2f13648b571da37e277540f00\" has failed: &{failed to analyze image: failed to run job manager: failed to run job: failed to create source analyzer=syft: could not fetch image 'uk.icr.io/sbu-pipeline/zap2docker-stable@sha256:6c9d3f2cc80470bb4b54fb4b402ff982905e5cb2f13648b571da37e277540f00': unable determine image source TBD}" func="github.com/cisco-open/kubei/runtime_scan/pkg/scanner.(*Scanner).HandleScanResults" file="/build/runtime_scan/pkg/scanner/scanner.go:399" scanner id=f880455f-e909-4fa5-8d7f-2c38a92d66c2
time="2022-04-27T16:09:01Z" level=info msg="Image scanned result has arrived. imageID=uk.icr.io/sbu-pipeline/zap2docker-stable@sha256:6c9d3f2cc80470bb4b54fb4b402ff982905e5cb2f13648b571da37e277540f00" func="github.com/cisco-open/kubei/runtime_scan/pkg/scanner.(*Scanner).waitForResult" file="/build/runtime_scan/pkg/scanner/job_managment.go:139" scanner id=f880455f-e909-4fa5-8d7f-2c38a92d66c2
time="2022-04-27T16:09:01Z" level=info msg="Vulnerabilities result was set for imageID \"uk.icr.io/sbu-pipeline/nginx-unprivileged@sha256:e86606220600fb7665e74453ca4e4b46a03215ea312e621c02809d2242a4bea2\"" func="github.com/cisco-open/kubei/runtime_scan/pkg/scanner.(*Scanner).HandleScanResults" file="/build/runtime_scan/pkg/scanner/scanner.go:392" scanner id=f880455f-e909-4fa5-8d7f-2c38a92d66c2
time="2022-04-27T16:09:01Z" level=warning msg="Vulnerabilities scan of imageID \"uk.icr.io/sbu-pipeline/nginx-unprivileged@sha256:e86606220600fb7665e74453ca4e4b46a03215ea312e621c02809d2242a4bea2\" has failed: &{failed to analyze image: failed to run job manager: failed to run job: failed to create source analyzer=syft: could not fetch image 'uk.icr.io/sbu-pipeline/nginx-unprivileged@sha256:e86606220600fb7665e74453ca4e4b46a03215ea312e621c02809d2242a4bea2': unable determine image source TBD}" func="github.com/cisco-open/kubei/runtime_scan/pkg/scanner.(*Scanner).HandleScanResults" file="/build/runtime_scan/pkg/scanner/scanner.go:399" scanner id=f880455f-e909-4fa5-8d7f-2c38a92d66c2
time="2022-04-27T16:09:01Z" level=info msg="Image scanned result has arrived. imageID=uk.icr.io/sbu-pipeline/nginx-unprivileged@sha256:e86606220600fb7665e74453ca4e4b46a03215ea312e621c02809d2242a4bea2" func="github.com/cisco-open/kubei/runtime_scan/pkg/scanner.(*Scanner).waitForResult" file="/build/runtime_scan/pkg/scanner/job_managment.go:139" scanner id=f880455f-e909-4fa5-8d7f-2c38a92d66c2
time="2022-04-27T16:09:01Z" level=info msg="All jobs has finished" func="github.com/cisco-open/kubei/runtime_scan/pkg/scanner.(*Scanner).jobBatchManagement" file="/build/runtime_scan/pkg/scanner/job_managment.go:93" scanner id=f880455f-e909-4fa5-8d7f-2c38a92d66c2
time="2022-04-27T16:09:01Z" level=info msg="Succeeded to apply runtime scan results. app ids=[], failures=[Failed to scan image \"uk.icr.io/sbu-pipeline/zap2docker-stable:2.11.1-sbu\".\nEffected pods: newzap-owaspzap-zap-b9dc5f9d7-qbm44/mattzap.\nReasons: failed to analyze image: failed to run job manager: failed to run job: failed to create source analyzer=syft: could not fetch image 'uk.icr.io/sbu-pipeline/zap2docker-stable@sha256:6c9d3f2cc80470bb4b54fb4b402ff982905e5cb2f13648b571da37e277540f00': unable determine image source. Failed to scan image \"uk.icr.io/sbu-pipeline/nginx-unprivileged:1.21.4-alpine-sbu\".\nEffected pods: newzap-owaspzap-zap-b9dc5f9d7-qbm44/mattzap.\nReasons: failed to analyze image: failed to run job manager: failed to run job: failed to create source analyzer=syft: could not fetch image 'uk.icr.io/sbu-pipeline/nginx-unprivileged@sha256:e86606220600fb7665e74453ca4e4b46a03215ea312e621c02809d2242a4bea2': unable determine image source.]" func="github.com/cisco-open/kubei/backend/pkg/rest.(*Server).startScan.func1" file="/build/backend/pkg/rest/runtime_scan_controller.go:255"

This is what shows in the UI:

Failed to scan image "uk.icr.io/sbu-pipeline/zap2docker-stable:2.11.1-sbu". Effected pods: newzap-owaspzap-zap-b9dc5f9d7-qbm44/mattzap. Reasons: failed to analyze image: failed to run job manager: failed to run job: failed to create source analyzer=syft: could not fetch image 'uk.icr.io/sbu-pipeline/zap2docker-stable@sha256:6c9d3f2cc80470bb4b54fb4b402ff982905e5cb2f13648b571da37e277540f00': unable determine image source.
Failed to scan image "uk.icr.io/sbu-pipeline/nginx-unprivileged:1.21.4-alpine-sbu". Effected pods: newzap-owaspzap-zap-b9dc5f9d7-qbm44/mattzap. Reasons: failed to analyze image: failed to run job manager: failed to run job: failed to create source analyzer=syft: could not fetch image 'uk.icr.io/sbu-pipeline/nginx-unprivileged@sha256:e86606220600fb7665e74453ca4e4b46a03215ea312e621c02809d2242a4bea2': unable determine image source.

from kubeclarity.

mtcolman avatar mtcolman commented on August 17, 2024

So I've now done:

$ podman pull alpine:3.14.3
WARN[0000] "/" is not a shared mount, this could cause issues or missing mounts with rootless containers
Resolved "alpine" as an alias (/etc/containers/registries.conf.d/000-shortnames.conf)
Trying to pull docker.io/library/alpine:3.14.3...
Getting image source signatures
Copying blob 97518928ae5f done
Copying config 0a97eee804 done
Writing manifest to image destination
Storing signatures
0a97eee8041e2b6c0e65abb2700b0705d0da5525ca69060b9e0bde8a3d17afdb

$ podman tag alpine:3.14.3 uk.icr.io/sbu-pipeline/alpinetest:3.14.3

$ podman push uk.icr.io/sbu-pipeline/alpinetest:3.14.3
Getting image source signatures
Copying blob 1a058d5342cc done
Copying config 0a97eee804 done
Writing manifest to image destination
Storing signatures

And cleared down the namespace:

$ oc get secret -n mattzap | grep sbu
sbu-pipeline               kubernetes.io/dockerconfigjson        1      22m

$ oc get pod -n mattzap
No resources found in mattzap namespace.

Then I deploy:

$ oc create -f new-alpine-deployment.yaml
deployment.apps/alpine created
serviceaccount/alpine created

And pod starts, using the sbu-pipeline secret as imagePull:

$ oc get pod -n mattzap
NAME                      READY   STATUS    RESTARTS   AGE
alpine-74d9957b8d-gn4ct   1/1     Running   0          8s

And this is the deployment.yaml:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: alpine
spec:
  selector:
    matchLabels:
      app: alpine
  replicas: 1
  template:
    metadata:
      labels:
        app: alpine
        version: "v3.14.3"
    spec:
      serviceAccountName: alpine
      imagePullSecrets:
      - name: sbu-pipeline
      containers:
        - name: alpine
          image: uk.icr.io/sbu-pipeline/alpinetest:3.14.3
          imagePullPolicy: IfNotPresent
          command: [ "sleep" ]
          args: [ "infinity" ]
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: alpine

kubeclarity logs show:

time="2022-04-27T16:28:57Z" level=info msg="Start scanning..." func="github.com/cisco-open/kubei/runtime_scan/pkg/scanner.(*Scanner).Scan" file="/build/runtime_scan/pkg/scanner/scanner.go:265" scanner id=531f4af4-60e3-4077-a0b1-5984b862e080
time="2022-04-27T16:28:57Z" level=debug msg="worker #1 halted" func="github.com/cisco-open/kubei/runtime_scan/pkg/scanner.(*Scanner).worker" file="/build/runtime_scan/pkg/scanner/job_managment.go:128" scanner id=f880455f-e909-4fa5-8d7f-2c38a92d66c2
time="2022-04-27T16:28:57Z" level=debug msg="worker #6 halted" func="github.com/cisco-open/kubei/runtime_scan/pkg/scanner.(*Scanner).worker" file="/build/runtime_scan/pkg/scanner/job_managment.go:128" scanner id=f880455f-e909-4fa5-8d7f-2c38a92d66c2
time="2022-04-27T16:28:57Z" level=debug msg="worker #8 halted" func="github.com/cisco-open/kubei/runtime_scan/pkg/scanner.(*Scanner).worker" file="/build/runtime_scan/pkg/scanner/job_managment.go:128" scanner id=f880455f-e909-4fa5-8d7f-2c38a92d66c2
time="2022-04-27T16:28:57Z" level=debug msg="worker #7 halted" func="github.com/cisco-open/kubei/runtime_scan/pkg/scanner.(*Scanner).worker" file="/build/runtime_scan/pkg/scanner/job_managment.go:128" scanner id=f880455f-e909-4fa5-8d7f-2c38a92d66c2
time="2022-04-27T16:28:57Z" level=debug msg="worker #9 halted" func="github.com/cisco-open/kubei/runtime_scan/pkg/scanner.(*Scanner).worker" file="/build/runtime_scan/pkg/scanner/job_managment.go:128" scanner id=f880455f-e909-4fa5-8d7f-2c38a92d66c2
time="2022-04-27T16:28:57Z" level=debug msg="worker #2 halted" func="github.com/cisco-open/kubei/runtime_scan/pkg/scanner.(*Scanner).worker" file="/build/runtime_scan/pkg/scanner/job_managment.go:128" scanner id=f880455f-e909-4fa5-8d7f-2c38a92d66c2
time="2022-04-27T16:28:57Z" level=debug msg="worker #5 halted" func="github.com/cisco-open/kubei/runtime_scan/pkg/scanner.(*Scanner).worker" file="/build/runtime_scan/pkg/scanner/job_managment.go:128" scanner id=f880455f-e909-4fa5-8d7f-2c38a92d66c2
time="2022-04-27T16:28:57Z" level=debug msg="worker #4 halted" func="github.com/cisco-open/kubei/runtime_scan/pkg/scanner.(*Scanner).worker" file="/build/runtime_scan/pkg/scanner/job_managment.go:128" scanner id=f880455f-e909-4fa5-8d7f-2c38a92d66c2
time="2022-04-27T16:28:57Z" level=debug msg="worker #3 halted" func="github.com/cisco-open/kubei/runtime_scan/pkg/scanner.(*Scanner).worker" file="/build/runtime_scan/pkg/scanner/job_managment.go:128" scanner id=f880455f-e909-4fa5-8d7f-2c38a92d66c2
time="2022-04-27T16:28:57Z" level=debug msg="No matching secret found." func=github.com/cisco-open/kubei/shared/pkg/utils/k8s.GetMatchingSecretName file="/build/shared/pkg/utils/k8s/k8s_utils.go:71"
time="2022-04-27T16:28:57Z" level=info msg="Total 1 unique images to scan" func="github.com/cisco-open/kubei/runtime_scan/pkg/scanner.(*Scanner).initScan" file="/build/runtime_scan/pkg/scanner/scanner.go:255" scanner id=531f4af4-60e3-4077-a0b1-5984b862e080
time="2022-04-27T16:28:57Z" level=debug msg="Created job=&Job{ObjectMeta:{scanner-alpinetest-1b21b6dc-1e25-45fa-a94f-5993c5109577  mattzap    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[app:scanner sidecar.istio.io/inject:false] map[] [] []  []},Spec:JobSpec{Parallelism:nil,Completions:nil,ActiveDeadlineSeconds:nil,Selector:nil,ManualSelector:nil,Template:{{      0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[app:scanner sidecar.istio.io/inject:false] map[] [] []  []} {[{tmp-volume {nil EmptyDirVolumeSource{Medium:,SizeLimit:<nil>,} nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil}}] [] [{vulnerability-scanner ghcr.io/cisco-open/kubeclarity-runtime-k8s-scanner:latest [] [scan --log-level debug]  [] [] [{RESULT_SERVICE_ADDR kubeclarity-kubeclarity.kubeclarity:8888 nil} {SBOM_DB_ADDR kubeclarity-kubeclarity-sbom-db.kubeclarity:8080 nil} {ANALYZER_LIST syft gomod nil} {ANALYZER_SCOPE squashed nil} {SCANNERS_LIST grype nil} {SCANNER_GRYPE_MODE REMOTE nil} {REGISTRY_SKIP_VERIFY_TLS false nil} {REGISTRY_USE_HTTP false nil} {SCANNER_REMOTE_GRYPE_SERVER_ADDRESS kubeclarity-kubeclarity-grype-server.kubeclarity:9991 nil} {SCANNER_REMOTE_GRYPE_SERVER_TIMEOUT 2m nil} {SCAN_UUID a3033112-a35d-4574-8532-4112bec76318 nil} {IMAGE_ID_TO_SCAN uk.icr.io/sbu-pipeline/alpinetest@sha256:213d60a32697894f9a1dbe3f309a5b5a737266a1b7866594937038514ebdc577 nil} {IMAGE_HASH_TO_SCAN 213d60a32697894f9a1dbe3f309a5b5a737266a1b7866594937038514ebdc577 nil} {IMAGE_NAME_TO_SCAN uk.icr.io/sbu-pipeline/alpinetest:3.14.3 nil}] {map[cpu:{{1000 -3} {<nil>}  DecimalSI} memory:{{1048576000 0} {<nil>}  BinarySI}] map[cpu:{{50 -3} {<nil>} 50m DecimalSI} memory:{{52428800 0} {<nil>} 50Mi BinarySI}]} [{tmp-volume false /tmp  <nil> }] [] nil nil nil nil   Always &SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[all],},Privileged:*false,SELinuxOptions:nil,RunAsUser:nil,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,} false false false}] [] Never <nil> <nil>  map[]   <nil>  false false false <nil> nil []   nil  [] []  <nil> nil [] <nil> <nil> <nil> map[] [] <nil> nil}},BackoffLimit:*0,TTLSecondsAfterFinished:*300,CompletionMode:nil,Suspend:nil,},Status:JobStatus{Conditions:[]JobCondition{},StartTime:<nil>,CompletionTime:<nil>,Active:0,Succeeded:0,Failed:0,CompletedIndexes:,UncountedTerminatedPods:nil,Ready:nil,},}" func="github.com/cisco-open/kubei/runtime_scan/pkg/scanner.(*Scanner).runJob" file="/build/runtime_scan/pkg/scanner/job_managment.go:164" scanner id=531f4af4-60e3-4077-a0b1-5984b862e080
time="2022-04-27T16:28:57Z" level=info msg="Running job mattzap/scanner-alpinetest-1b21b6dc-1e25-45fa-a94f-5993c5109577 to scan imageID uk.icr.io/sbu-pipeline/alpinetest@sha256:213d60a32697894f9a1dbe3f309a5b5a737266a1b7866594937038514ebdc577" func="github.com/cisco-open/kubei/runtime_scan/pkg/scanner.(*Scanner).runJob" file="/build/runtime_scan/pkg/scanner/job_managment.go:166" scanner id=531f4af4-60e3-4077-a0b1-5984b862e080
time="2022-04-27T16:28:57Z" level=info msg="Waiting for result. imageID=uk.icr.io/sbu-pipeline/alpinetest@sha256:213d60a32697894f9a1dbe3f309a5b5a737266a1b7866594937038514ebdc577" func="github.com/cisco-open/kubei/runtime_scan/pkg/scanner.(*Scanner).waitForResult" file="/build/runtime_scan/pkg/scanner/job_managment.go:135" scanner id=531f4af4-60e3-4077-a0b1-5984b862e080
time="2022-04-27T16:29:06Z" level=info msg="Vulnerabilities result was set for imageID \"uk.icr.io/sbu-pipeline/alpinetest@sha256:213d60a32697894f9a1dbe3f309a5b5a737266a1b7866594937038514ebdc577\"" func="github.com/cisco-open/kubei/runtime_scan/pkg/scanner.(*Scanner).HandleScanResults" file="/build/runtime_scan/pkg/scanner/scanner.go:392" scanner id=531f4af4-60e3-4077-a0b1-5984b862e080
time="2022-04-27T16:29:06Z" level=warning msg="Vulnerabilities scan of imageID \"uk.icr.io/sbu-pipeline/alpinetest@sha256:213d60a32697894f9a1dbe3f309a5b5a737266a1b7866594937038514ebdc577\" has failed: &{failed to analyze image: failed to run job manager: failed to run job: failed to create source analyzer=syft: could not fetch image 'uk.icr.io/sbu-pipeline/alpinetest@sha256:213d60a32697894f9a1dbe3f309a5b5a737266a1b7866594937038514ebdc577': unable determine image source TBD}" func="github.com/cisco-open/kubei/runtime_scan/pkg/scanner.(*Scanner).HandleScanResults" file="/build/runtime_scan/pkg/scanner/scanner.go:399" scanner id=531f4af4-60e3-4077-a0b1-5984b862e080
time="2022-04-27T16:29:06Z" level=info msg="Image scanned result has arrived. imageID=uk.icr.io/sbu-pipeline/alpinetest@sha256:213d60a32697894f9a1dbe3f309a5b5a737266a1b7866594937038514ebdc577" func="github.com/cisco-open/kubei/runtime_scan/pkg/scanner.(*Scanner).waitForResult" file="/build/runtime_scan/pkg/scanner/job_managment.go:139" scanner id=531f4af4-60e3-4077-a0b1-5984b862e080
time="2022-04-27T16:29:06Z" level=info msg="All jobs has finished" func="github.com/cisco-open/kubei/runtime_scan/pkg/scanner.(*Scanner).jobBatchManagement" file="/build/runtime_scan/pkg/scanner/job_managment.go:93" scanner id=531f4af4-60e3-4077-a0b1-5984b862e080
time="2022-04-27T16:29:06Z" level=info msg="Succeeded to apply runtime scan results. app ids=[], failures=[Failed to scan image \"uk.icr.io/sbu-pipeline/alpinetest:3.14.3\".\nEffected pods: alpine-74d9957b8d-gn4ct/mattzap.\nReasons: failed to analyze image: failed to run job manager: failed to run job: failed to create source analyzer=syft: could not fetch image 'uk.icr.io/sbu-pipeline/alpinetest@sha256:213d60a32697894f9a1dbe3f309a5b5a737266a1b7866594937038514ebdc577': unable determine image source.]" func="github.com/cisco-open/kubei/backend/pkg/rest.(*Server).startScan.func1" file="/build/backend/pkg/rest/runtime_scan_controller.go:255"

Scanner logs show (had to grab from a subsequent run as a I forgot):

$ oc logs scanner-alpinetest-1898289c-def5-4726-88ab-ad614ba7549--1-x48bk
time="2022-04-27T16:42:26Z" level=debug msg="Credentials not found. image name=uk.icr.io/sbu-pipeline/alpinetest@sha256:213d60a32697894f9a1dbe3f309a5b5a737266a1b7866594937038514ebdc577." func="github.com/cisco-open/kubei/shared/pkg/utils/creds.(*CredExtractor).GetCredentials" file="/build/shared/pkg/utils/creds/extractor.go:78"
time="2022-04-27T16:42:26Z" level=info msg="\n\nconfig={\"ResultServiceAddress\":\"kubeclarity-kubeclarity.kubeclarity:8888\",\"SBOMDBAddress\":\"kubeclarity-kubeclarity-sbom-db.kubeclarity:8080\",\"ImageIDToScan\":\"uk.icr.io/sbu-pipeline/alpinetest@sha256:213d60a32697894f9a1dbe3f309a5b5a737266a1b7866594937038514ebdc577\",\"ImageHashToScan\":\"213d60a32697894f9a1dbe3f309a5b5a737266a1b7866594937038514ebdc577\",\"ScanUUID\":\"c7d0e605-31bd-4073-8075-e85ab9d2845d\",\"RegistryInsecure\":false,\"SharedConfig\":{\"Registry\":{\"skip-verify-tls\":false,\"use-http\":false,\"auths\":[{\"authority\":\"\"}]},\"Analyzer\":{\"OutputFormat\":\"cyclonedx\",\"AnalyzerList\":[\"syft\",\"gomod\"],\"Scope\":\"squashed\"},\"Scanner\":{\"ScannersList\":[\"grype\"],\"GrypeConfig\":{\"UpdateDB\":true,\"DBRootDir\":\"/tmp/\",\"ListingURL\":\"https://toolbox-data.anchore.io/grype/databases/listing.json\",\"Scope\":\"Squashed\",\"GrypeServerAddress\":\"kubeclarity-kubeclarity-grype-server.kubeclarity:9991\",\"GrypeServerTimeout\":120000000000,\"Mode\":\"REMOTE\"},\"DependencyTrackConfig\":{\"host\":\"dependency-track-apiserver.dependency-track\",\"project-name\":\"\",\"project-version\":\"\",\"should-delete-project\":true,\"disable-tls\":false,\"insecure-skip-verify\":true,\"fetch-vulnerabilities-retry-count\":5,\"fetch-vulnerabilities-retry-sleep\":30000000000}},\"LocalImageScan\":false},\"ImageNameToScan\":\"uk.icr.io/sbu-pipeline/alpinetest:3.14.3\"}\n\n" func=github.com/cisco-open/kubei/runtime_k8s_scanner/pkg/config.LoadConfig file="/build/runtime_k8s_scanner/pkg/config/config.go:61"
time="2022-04-27T16:42:26Z" level=info msg="Runtime K8s Scanner job is running. imageID=uk.icr.io/sbu-pipeline/alpinetest@sha256:213d60a32697894f9a1dbe3f309a5b5a737266a1b7866594937038514ebdc577, hash=213d60a32697894f9a1dbe3f309a5b5a737266a1b7866594937038514ebdc577" func=github.com/cisco-open/kubei/runtime_k8s_scanner/pkg/scanner.Run file="/build/runtime_k8s_scanner/pkg/scanner/scanner.go:51"
time="2022-04-27T16:42:26Z" level=info msg="Called syft analyzer on source registry:uk.icr.io/sbu-pipeline/alpinetest@sha256:213d60a32697894f9a1dbe3f309a5b5a737266a1b7866594937038514ebdc577" analyzer=syft component=analyzer image-id="uk.icr.io/sbu-pipeline/alpinetest@sha256:213d60a32697894f9a1dbe3f309a5b5a737266a1b7866594937038514ebdc577" scan-uuid=c7d0e605-31bd-4073-8075-e85ab9d2845d
time="2022-04-27T16:42:26Z" level=error msg="failed to analyze image: failed to run job manager: failed to run job: failed to create source analyzer=syft: could not fetch image 'uk.icr.io/sbu-pipeline/alpinetest@sha256:213d60a32697894f9a1dbe3f309a5b5a737266a1b7866594937038514ebdc577': unable determine image source" image-id="uk.icr.io/sbu-pipeline/alpinetest@sha256:213d60a32697894f9a1dbe3f309a5b5a737266a1b7866594937038514ebdc577" scan-uuid=c7d0e605-31bd-4073-8075-e85ab9d2845d
time="2022-04-27T16:42:26Z" level=info msg="Runtime K8s Scanner job was done." func=github.com/cisco-open/kubei/runtime_k8s_scanner/pkg/scanner.Run file="/build/runtime_k8s_scanner/pkg/scanner/scanner.go:64"

from kubeclarity.

akpsgit avatar akpsgit commented on August 17, 2024

@mtcolman, thank you for the detailed information.
There are several checks that are being done for the image pull secret to make sure that it can be used:
https://github.com/cisco-open/kubei/blob/ea46fc3cbe0dc819e57c18511dc0c96bb24e48b2/shared/pkg/utils/k8s/k8s_utils.go#L49

From the code, I see that if the password and the username are both empty, the image pull secret is being ignore, disregarding the additional Auth property.

Can you please check if that's indeed the case in the sbu-pipeline secret.
There are some additional checks that the auth corresponds to the image, that might also cause the mismatch, but i'm doubted as K8s manages to pull the image and it uses the same pull secret code as we do.

The matching logic is tested in this unit test:
https://github.com/cisco-open/kubei/blob/ea46fc3cbe0dc819e57c18511dc0c96bb24e48b2/shared/pkg/utils/k8s/k8s_utils_test.go#L65
maybe you can add your specific structure to the unit test to see that it's matched. Also, we can test it if you can strip all sensitive information (e.g. replace with XXX) and share the secret json.

Thanks!!!

from kubeclarity.

mtcolman avatar mtcolman commented on August 17, 2024

@akpsgit the username and password are both supplied in sbu-pipeline and this is validated by the fact the image can be pulled when the pod is deployed. - this is shown in the below comment as well for evidence.

from kubeclarity.

mtcolman avatar mtcolman commented on August 17, 2024

Here is the secret:

$ oc get secret sbu-pipeline -n mattzap -oyaml
apiVersion: v1
data:
  .dockerconfigjson: <base64-redacted-by-me>
kind: Secret
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"v1","data":{".dockerconfigjson":"<base64-redacted-by-me>"},"kind":"Secret","metadata":{"annotations":{},"creationTimestamp":null,"name":"sbu-pipeline","namespace":"mattzap"},"type":"kubernetes.io/dockerconfigjson"}
  creationTimestamp: "2022-04-27T16:05:45Z"
  name: sbu-pipeline
  namespace: mattzap
  resourceVersion: "706789537"
  uid: da8364a4-d023-4281-a2a9-22b2b88bfaac
type: kubernetes.io/dockerconfigjson

Here is me showing the decoded base64:

$ echo -n <base64-redacted-by-me> | base64 -d
{"auths":{"uk.icr.io/sbu-pipeline":{"username":"iamapikey","password":"<redacted-by-me>","email":"unused","auth":"<base64-encoded-iamapikey:password>"}}}

from kubeclarity.

FrimIdan avatar FrimIdan commented on August 17, 2024

@mmurhamm sorry for the inconvenience, it seems like that if you will use registry name uk.icr.io or the full target name uk.icr.io/sbu-pipeline/alpinetest in the image pull secret it will work.

We need to dig more to understand what was changes between the old logic and the new one.

{"auths":{"uk.icr.io":{"username":"iamapikey","password":"<redacted-by-me>","email":"unused","auth":"<base64-encoded-iamapikey:password>"}}}

OR

{"auths":{"uk.icr.io/sbu-pipeline/alpinetest":{"username":"iamapikey","password":"<redacted-by-me>","email":"unused","auth":"<base64-encoded-iamapikey:password>"}}}

from kubeclarity.

FrimIdan avatar FrimIdan commented on August 17, 2024

Hi @mtcolman v2.2.0 just released, can we close the issue?

from kubeclarity.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.