Comments (15)
Hi @mtcolman found the following commit in go-containerregistry google/go-containerregistry@892d7a8 pushed 14 days ago.
It should fix that issue here, we will update go.mod and will add a test case.
from kubeclarity.
@FrimIdan tested and looks good, yes I believe you can close. Thanks for speedy fix.
from kubeclarity.
The expected behavior described above actually worked with kubei (1.0.15); it would be helpful to have somewhat more detailed documentation about how kubeclarity works internally and how it differs from kubei. Thanks.
from kubeclarity.
I increased the log-level and have also now seen this:
{IMAGE_NAME_TO_SCAN uk.icr.io/sbu-pipeline/zap2docker-stable:2.11.1-sbu nil} {K8S_IMAGE_PULL_SECRET &EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:&SecretKeySelector{LocalObjectReference:LocalObjectReference{Name:basic-regcred,},Key:.dockerconfigjson,Optional:nil,},}}]
and this:
time="2022-04-26T18:19:50Z" level=debug msg="No matching secret found." func=github.com/cisco-open/kubei/shared/pkg/utils/k8s.GetMatchingSecretName file="/build/shared/pkg/utils/k8s/k8s_utils.go:71"
Which I believe relates to lines 202 and 231 of this code whereby the image pull secret is supposedly obtained and then the matching secret obtained as well.
from kubeclarity.
Hi @mtcolman @mmurhamm, thanks for the information. We'll add documents about the architectural changes between Kubei 1 and a Kubei 2 to make it more clear. Regarding the image pull secret, the main difference is that we moved from using DockerKeyRing https://github.com/cisco-open/kubei/blob/1.0.16/pkg/utils/k8s/k8s_utils.go#L57 to use
"github.com/google/go-containerregistry/pkg/authn/kubernetes:
https://github.com/cisco-open/kubei/blob/3217eb2e7da5b7d05952dc9711cc1ad9efd2b0cb/shared/pkg/utils/k8s/k8s_utils.go#L75
From the provided logs, looks like the basic-regcred secret should be used (no ImagePullSecret on the pod), right?
from kubeclarity.
@akpsgit thanks for your reply. The imagePullSecret
is in the pod to be scanned, so it would expect it to use that rather than falling back to basic-regcred
:
$ oc get pod matt-zap-owaspzap-zap-66cb56cdc4-wkxlw -n sbu-dev -oyaml | grep PullSecrets -A2
imagePullSecrets:
- name: sbu-pipeline
Secret is present in that namespace as well:
$ oc get secret -n sbu-dev | grep sbu
sbu-pipeline kubernetes.io/dockerconfigjson 1 152d
from kubeclarity.
Is matt-zap-owaspzap-zap-66cb56cdc4-wkxlw is the only pod that contains the uk.icr.io/sbu-pipeline/zap2docker-stable:2.11 image? I'm asking because we create a scanner per image, and take the details from the first pod in the list that contains this image (might not have the pull secret) - small chance but worth checking:
https://github.com/cisco-open/kubei/blob/3217eb2e7da5b7d05952dc9711cc1ad9efd2b0cb/runtime_scan/pkg/scanner/job_managment.go#L262
If there is no other pod that uses this image, worth trying to start this pod in a separate namespace and scan only it to get more coherent logs in KubeClarity backend (where the job management is running). If nothing new comes up, can you please attach the "oc get pod matt-zap-owaspzap-zap-66cb56cdc4-wkxlw -n sbu-dev -oyaml" output (with stripped sensitive/identifying values) so we can try to recreate and debug it?
from kubeclarity.
I can confirm it is the only pod that uses this image:
$ oc get pod -n sbu-dev -oyaml | grep zap2docker -A5 -B5
- name: ZAP_APIKEY
valueFrom:
secretKeyRef:
key: zap-api-key
name: matt-zap-zap-key
image: uk.icr.io/sbu-pipeline/zap2docker-stable:2.11.1-sbu
imagePullPolicy: IfNotPresent
name: owaspzap
ports:
- containerPort: 8081
protocol: TCP
--
lastTransitionTime: "2022-03-31T03:09:26Z"
status: "True"
type: PodScheduled
containerStatuses:
- containerID: cri-o://d15f80077fc96558e7a0e36202d8b55e4051f21d53668abac9614b6ada558707
image: uk.icr.io/sbu-pipeline/zap2docker-stable:2.11.1-sbu
imageID: uk.icr.io/sbu-pipeline/zap2docker-stable@sha256:6c9d3f2cc80470bb4b54fb4b402ff982905e5cb2f13648b571da37e277540f00
lastState:
terminated:
containerID: cri-o://4897b696c88bcb0bf8d067d83af59001c630e481b230c569107f6bb3e44919c7
exitCode: 137
finishedAt: "2022-04-27T14:18:37Z"
from kubeclarity.
Created brand new namespace mattzap
and deployed pod. That via UI, selected namespace and ran scan:
$ oc get pod -n mattzap
NAME READY STATUS RESTARTS AGE
newzap-owaspzap-zap-b9dc5f9d7-qbm44 2/2 Running 0 2m32s
scanner-nginx-unprivileged-e34e22b6-7eab-4d4a-a45a-fc3--1-sg4bw 0/1 Completed 0 54s
scanner-zap2docker-stable-78eba9a6-6014-49bb-8331-2dac--1-pvndz 0/1 Completed 0 54s
Scan results are:
time="2022-04-27T16:08:52Z" level=info msg="Start scanning..." func="github.com/cisco-open/kubei/runtime_scan/pkg/scanner.(*Scanner).Scan" file="/build/runtime_scan/pkg/scanner/scanner.go:265" scanner id=f880455f-e909-4fa5-8d7f-2c38a92d66c2
time="2022-04-27T16:08:52Z" level=debug msg="No matching secret found." func=github.com/cisco-open/kubei/shared/pkg/utils/k8s.GetMatchingSecretName file="/build/shared/pkg/utils/k8s/k8s_utils.go:71"
time="2022-04-27T16:08:52Z" level=debug msg="No matching secret found." func=github.com/cisco-open/kubei/shared/pkg/utils/k8s.GetMatchingSecretName file="/build/shared/pkg/utils/k8s/k8s_utils.go:71"
time="2022-04-27T16:08:52Z" level=info msg="Total 2 unique images to scan" func="github.com/cisco-open/kubei/runtime_scan/pkg/scanner.(*Scanner).initScan" file="/build/runtime_scan/pkg/scanner/scanner.go:255" scanner id=f880455f-e909-4fa5-8d7f-2c38a92d66c2
time="2022-04-27T16:08:52Z" level=debug msg="Created job=&Job{ObjectMeta:{scanner-nginx-unprivileged-e34e22b6-7eab-4d4a-a45a-fc3366394ca3 mattzap 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[app:scanner sidecar.istio.io/inject:false] map[] [] [] []},Spec:JobSpec{Parallelism:nil,Completions:nil,ActiveDeadlineSeconds:nil,Selector:nil,ManualSelector:nil,Template:{{ 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[app:scanner sidecar.istio.io/inject:false] map[] [] [] []} {[{tmp-volume {nil EmptyDirVolumeSource{Medium:,SizeLimit:<nil>,} nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil}}] [] [{vulnerability-scanner ghcr.io/cisco-open/kubeclarity-runtime-k8s-scanner:latest [] [scan --log-level debug] [] [] [{RESULT_SERVICE_ADDR kubeclarity-kubeclarity.kubeclarity:8888 nil} {SBOM_DB_ADDR kubeclarity-kubeclarity-sbom-db.kubeclarity:8080 nil} {ANALYZER_LIST syft gomod nil} {ANALYZER_SCOPE squashed nil} {SCANNERS_LIST grype nil} {SCANNER_GRYPE_MODE REMOTE nil} {REGISTRY_SKIP_VERIFY_TLS false nil} {REGISTRY_USE_HTTP false nil} {SCANNER_REMOTE_GRYPE_SERVER_ADDRESS kubeclarity-kubeclarity-grype-server.kubeclarity:9991 nil} {SCANNER_REMOTE_GRYPE_SERVER_TIMEOUT 2m nil} {SCAN_UUID 4a32c0a3-296b-440c-839c-a0f1478e1098 nil} {IMAGE_ID_TO_SCAN uk.icr.io/sbu-pipeline/nginx-unprivileged@sha256:e86606220600fb7665e74453ca4e4b46a03215ea312e621c02809d2242a4bea2 nil} {IMAGE_HASH_TO_SCAN e86606220600fb7665e74453ca4e4b46a03215ea312e621c02809d2242a4bea2 nil} {IMAGE_NAME_TO_SCAN uk.icr.io/sbu-pipeline/nginx-unprivileged:1.21.4-alpine-sbu nil}] {map[cpu:{{1000 -3} {<nil>} DecimalSI} memory:{{1048576000 0} {<nil>} BinarySI}] map[cpu:{{50 -3} {<nil>} 50m DecimalSI} memory:{{52428800 0} {<nil>} 50Mi BinarySI}]} [{tmp-volume false /tmp <nil> }] [] nil nil nil nil Always &SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[all],},Privileged:*false,SELinuxOptions:nil,RunAsUser:nil,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,} false false false}] [] Never <nil> <nil> map[] <nil> false false false <nil> nil [] nil [] [] <nil> nil [] <nil> <nil> <nil> map[] [] <nil> nil}},BackoffLimit:*0,TTLSecondsAfterFinished:*300,CompletionMode:nil,Suspend:nil,},Status:JobStatus{Conditions:[]JobCondition{},StartTime:<nil>,CompletionTime:<nil>,Active:0,Succeeded:0,Failed:0,CompletedIndexes:,UncountedTerminatedPods:nil,Ready:nil,},}" func="github.com/cisco-open/kubei/runtime_scan/pkg/scanner.(*Scanner).runJob" file="/build/runtime_scan/pkg/scanner/job_managment.go:164" scanner id=f880455f-e909-4fa5-8d7f-2c38a92d66c2
time="2022-04-27T16:08:52Z" level=info msg="Running job mattzap/scanner-nginx-unprivileged-e34e22b6-7eab-4d4a-a45a-fc3366394ca3 to scan imageID uk.icr.io/sbu-pipeline/nginx-unprivileged@sha256:e86606220600fb7665e74453ca4e4b46a03215ea312e621c02809d2242a4bea2" func="github.com/cisco-open/kubei/runtime_scan/pkg/scanner.(*Scanner).runJob" file="/build/runtime_scan/pkg/scanner/job_managment.go:166" scanner id=f880455f-e909-4fa5-8d7f-2c38a92d66c2
time="2022-04-27T16:08:52Z" level=debug msg="Created job=&Job{ObjectMeta:{scanner-zap2docker-stable-78eba9a6-6014-49bb-8331-2daca6ce9373 mattzap 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[app:scanner sidecar.istio.io/inject:false] map[] [] [] []},Spec:JobSpec{Parallelism:nil,Completions:nil,ActiveDeadlineSeconds:nil,Selector:nil,ManualSelector:nil,Template:{{ 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[app:scanner sidecar.istio.io/inject:false] map[] [] [] []} {[{tmp-volume {nil EmptyDirVolumeSource{Medium:,SizeLimit:<nil>,} nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil}}] [] [{vulnerability-scanner ghcr.io/cisco-open/kubeclarity-runtime-k8s-scanner:latest [] [scan --log-level debug] [] [] [{RESULT_SERVICE_ADDR kubeclarity-kubeclarity.kubeclarity:8888 nil} {SBOM_DB_ADDR kubeclarity-kubeclarity-sbom-db.kubeclarity:8080 nil} {ANALYZER_LIST syft gomod nil} {ANALYZER_SCOPE squashed nil} {SCANNERS_LIST grype nil} {SCANNER_GRYPE_MODE REMOTE nil} {REGISTRY_SKIP_VERIFY_TLS false nil} {REGISTRY_USE_HTTP false nil} {SCANNER_REMOTE_GRYPE_SERVER_ADDRESS kubeclarity-kubeclarity-grype-server.kubeclarity:9991 nil} {SCANNER_REMOTE_GRYPE_SERVER_TIMEOUT 2m nil} {SCAN_UUID c8321846-3f3f-44ea-8bf7-e6ebdb50deb1 nil} {IMAGE_ID_TO_SCAN uk.icr.io/sbu-pipeline/zap2docker-stable@sha256:6c9d3f2cc80470bb4b54fb4b402ff982905e5cb2f13648b571da37e277540f00 nil} {IMAGE_HASH_TO_SCAN 6c9d3f2cc80470bb4b54fb4b402ff982905e5cb2f13648b571da37e277540f00 nil} {IMAGE_NAME_TO_SCAN uk.icr.io/sbu-pipeline/zap2docker-stable:2.11.1-sbu nil}] {map[cpu:{{1000 -3} {<nil>} DecimalSI} memory:{{1048576000 0} {<nil>} BinarySI}] map[cpu:{{50 -3} {<nil>} 50m DecimalSI} memory:{{52428800 0} {<nil>} 50Mi BinarySI}]} [{tmp-volume false /tmp <nil> }] [] nil nil nil nil Always &SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[all],},Privileged:*false,SELinuxOptions:nil,RunAsUser:nil,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,} false false false}] [] Never <nil> <nil> map[] <nil> false false false <nil> nil [] nil [] [] <nil> nil [] <nil> <nil> <nil> map[] [] <nil> nil}},BackoffLimit:*0,TTLSecondsAfterFinished:*300,CompletionMode:nil,Suspend:nil,},Status:JobStatus{Conditions:[]JobCondition{},StartTime:<nil>,CompletionTime:<nil>,Active:0,Succeeded:0,Failed:0,CompletedIndexes:,UncountedTerminatedPods:nil,Ready:nil,},}" func="github.com/cisco-open/kubei/runtime_scan/pkg/scanner.(*Scanner).runJob" file="/build/runtime_scan/pkg/scanner/job_managment.go:164" scanner id=f880455f-e909-4fa5-8d7f-2c38a92d66c2
time="2022-04-27T16:08:52Z" level=info msg="Running job mattzap/scanner-zap2docker-stable-78eba9a6-6014-49bb-8331-2daca6ce9373 to scan imageID uk.icr.io/sbu-pipeline/zap2docker-stable@sha256:6c9d3f2cc80470bb4b54fb4b402ff982905e5cb2f13648b571da37e277540f00" func="github.com/cisco-open/kubei/runtime_scan/pkg/scanner.(*Scanner).runJob" file="/build/runtime_scan/pkg/scanner/job_managment.go:166" scanner id=f880455f-e909-4fa5-8d7f-2c38a92d66c2
time="2022-04-27T16:08:53Z" level=info msg="Waiting for result. imageID=uk.icr.io/sbu-pipeline/nginx-unprivileged@sha256:e86606220600fb7665e74453ca4e4b46a03215ea312e621c02809d2242a4bea2" func="github.com/cisco-open/kubei/runtime_scan/pkg/scanner.(*Scanner).waitForResult" file="/build/runtime_scan/pkg/scanner/job_managment.go:135" scanner id=f880455f-e909-4fa5-8d7f-2c38a92d66c2
time="2022-04-27T16:08:53Z" level=info msg="Waiting for result. imageID=uk.icr.io/sbu-pipeline/zap2docker-stable@sha256:6c9d3f2cc80470bb4b54fb4b402ff982905e5cb2f13648b571da37e277540f00" func="github.com/cisco-open/kubei/runtime_scan/pkg/scanner.(*Scanner).waitForResult" file="/build/runtime_scan/pkg/scanner/job_managment.go:135" scanner id=f880455f-e909-4fa5-8d7f-2c38a92d66c2
time="2022-04-27T16:09:01Z" level=info msg="Vulnerabilities result was set for imageID \"uk.icr.io/sbu-pipeline/zap2docker-stable@sha256:6c9d3f2cc80470bb4b54fb4b402ff982905e5cb2f13648b571da37e277540f00\"" func="github.com/cisco-open/kubei/runtime_scan/pkg/scanner.(*Scanner).HandleScanResults" file="/build/runtime_scan/pkg/scanner/scanner.go:392" scanner id=f880455f-e909-4fa5-8d7f-2c38a92d66c2
time="2022-04-27T16:09:01Z" level=warning msg="Vulnerabilities scan of imageID \"uk.icr.io/sbu-pipeline/zap2docker-stable@sha256:6c9d3f2cc80470bb4b54fb4b402ff982905e5cb2f13648b571da37e277540f00\" has failed: &{failed to analyze image: failed to run job manager: failed to run job: failed to create source analyzer=syft: could not fetch image 'uk.icr.io/sbu-pipeline/zap2docker-stable@sha256:6c9d3f2cc80470bb4b54fb4b402ff982905e5cb2f13648b571da37e277540f00': unable determine image source TBD}" func="github.com/cisco-open/kubei/runtime_scan/pkg/scanner.(*Scanner).HandleScanResults" file="/build/runtime_scan/pkg/scanner/scanner.go:399" scanner id=f880455f-e909-4fa5-8d7f-2c38a92d66c2
time="2022-04-27T16:09:01Z" level=info msg="Image scanned result has arrived. imageID=uk.icr.io/sbu-pipeline/zap2docker-stable@sha256:6c9d3f2cc80470bb4b54fb4b402ff982905e5cb2f13648b571da37e277540f00" func="github.com/cisco-open/kubei/runtime_scan/pkg/scanner.(*Scanner).waitForResult" file="/build/runtime_scan/pkg/scanner/job_managment.go:139" scanner id=f880455f-e909-4fa5-8d7f-2c38a92d66c2
time="2022-04-27T16:09:01Z" level=info msg="Vulnerabilities result was set for imageID \"uk.icr.io/sbu-pipeline/nginx-unprivileged@sha256:e86606220600fb7665e74453ca4e4b46a03215ea312e621c02809d2242a4bea2\"" func="github.com/cisco-open/kubei/runtime_scan/pkg/scanner.(*Scanner).HandleScanResults" file="/build/runtime_scan/pkg/scanner/scanner.go:392" scanner id=f880455f-e909-4fa5-8d7f-2c38a92d66c2
time="2022-04-27T16:09:01Z" level=warning msg="Vulnerabilities scan of imageID \"uk.icr.io/sbu-pipeline/nginx-unprivileged@sha256:e86606220600fb7665e74453ca4e4b46a03215ea312e621c02809d2242a4bea2\" has failed: &{failed to analyze image: failed to run job manager: failed to run job: failed to create source analyzer=syft: could not fetch image 'uk.icr.io/sbu-pipeline/nginx-unprivileged@sha256:e86606220600fb7665e74453ca4e4b46a03215ea312e621c02809d2242a4bea2': unable determine image source TBD}" func="github.com/cisco-open/kubei/runtime_scan/pkg/scanner.(*Scanner).HandleScanResults" file="/build/runtime_scan/pkg/scanner/scanner.go:399" scanner id=f880455f-e909-4fa5-8d7f-2c38a92d66c2
time="2022-04-27T16:09:01Z" level=info msg="Image scanned result has arrived. imageID=uk.icr.io/sbu-pipeline/nginx-unprivileged@sha256:e86606220600fb7665e74453ca4e4b46a03215ea312e621c02809d2242a4bea2" func="github.com/cisco-open/kubei/runtime_scan/pkg/scanner.(*Scanner).waitForResult" file="/build/runtime_scan/pkg/scanner/job_managment.go:139" scanner id=f880455f-e909-4fa5-8d7f-2c38a92d66c2
time="2022-04-27T16:09:01Z" level=info msg="All jobs has finished" func="github.com/cisco-open/kubei/runtime_scan/pkg/scanner.(*Scanner).jobBatchManagement" file="/build/runtime_scan/pkg/scanner/job_managment.go:93" scanner id=f880455f-e909-4fa5-8d7f-2c38a92d66c2
time="2022-04-27T16:09:01Z" level=info msg="Succeeded to apply runtime scan results. app ids=[], failures=[Failed to scan image \"uk.icr.io/sbu-pipeline/zap2docker-stable:2.11.1-sbu\".\nEffected pods: newzap-owaspzap-zap-b9dc5f9d7-qbm44/mattzap.\nReasons: failed to analyze image: failed to run job manager: failed to run job: failed to create source analyzer=syft: could not fetch image 'uk.icr.io/sbu-pipeline/zap2docker-stable@sha256:6c9d3f2cc80470bb4b54fb4b402ff982905e5cb2f13648b571da37e277540f00': unable determine image source. Failed to scan image \"uk.icr.io/sbu-pipeline/nginx-unprivileged:1.21.4-alpine-sbu\".\nEffected pods: newzap-owaspzap-zap-b9dc5f9d7-qbm44/mattzap.\nReasons: failed to analyze image: failed to run job manager: failed to run job: failed to create source analyzer=syft: could not fetch image 'uk.icr.io/sbu-pipeline/nginx-unprivileged@sha256:e86606220600fb7665e74453ca4e4b46a03215ea312e621c02809d2242a4bea2': unable determine image source.]" func="github.com/cisco-open/kubei/backend/pkg/rest.(*Server).startScan.func1" file="/build/backend/pkg/rest/runtime_scan_controller.go:255"
This is what shows in the UI:
Failed to scan image "uk.icr.io/sbu-pipeline/zap2docker-stable:2.11.1-sbu". Effected pods: newzap-owaspzap-zap-b9dc5f9d7-qbm44/mattzap. Reasons: failed to analyze image: failed to run job manager: failed to run job: failed to create source analyzer=syft: could not fetch image 'uk.icr.io/sbu-pipeline/zap2docker-stable@sha256:6c9d3f2cc80470bb4b54fb4b402ff982905e5cb2f13648b571da37e277540f00': unable determine image source.
Failed to scan image "uk.icr.io/sbu-pipeline/nginx-unprivileged:1.21.4-alpine-sbu". Effected pods: newzap-owaspzap-zap-b9dc5f9d7-qbm44/mattzap. Reasons: failed to analyze image: failed to run job manager: failed to run job: failed to create source analyzer=syft: could not fetch image 'uk.icr.io/sbu-pipeline/nginx-unprivileged@sha256:e86606220600fb7665e74453ca4e4b46a03215ea312e621c02809d2242a4bea2': unable determine image source.
from kubeclarity.
So I've now done:
$ podman pull alpine:3.14.3
WARN[0000] "/" is not a shared mount, this could cause issues or missing mounts with rootless containers
Resolved "alpine" as an alias (/etc/containers/registries.conf.d/000-shortnames.conf)
Trying to pull docker.io/library/alpine:3.14.3...
Getting image source signatures
Copying blob 97518928ae5f done
Copying config 0a97eee804 done
Writing manifest to image destination
Storing signatures
0a97eee8041e2b6c0e65abb2700b0705d0da5525ca69060b9e0bde8a3d17afdb
$ podman tag alpine:3.14.3 uk.icr.io/sbu-pipeline/alpinetest:3.14.3
$ podman push uk.icr.io/sbu-pipeline/alpinetest:3.14.3
Getting image source signatures
Copying blob 1a058d5342cc done
Copying config 0a97eee804 done
Writing manifest to image destination
Storing signatures
And cleared down the namespace:
$ oc get secret -n mattzap | grep sbu
sbu-pipeline kubernetes.io/dockerconfigjson 1 22m
$ oc get pod -n mattzap
No resources found in mattzap namespace.
Then I deploy:
$ oc create -f new-alpine-deployment.yaml
deployment.apps/alpine created
serviceaccount/alpine created
And pod starts, using the sbu-pipeline
secret as imagePull:
$ oc get pod -n mattzap
NAME READY STATUS RESTARTS AGE
alpine-74d9957b8d-gn4ct 1/1 Running 0 8s
And this is the deployment.yaml:
apiVersion: apps/v1
kind: Deployment
metadata:
name: alpine
spec:
selector:
matchLabels:
app: alpine
replicas: 1
template:
metadata:
labels:
app: alpine
version: "v3.14.3"
spec:
serviceAccountName: alpine
imagePullSecrets:
- name: sbu-pipeline
containers:
- name: alpine
image: uk.icr.io/sbu-pipeline/alpinetest:3.14.3
imagePullPolicy: IfNotPresent
command: [ "sleep" ]
args: [ "infinity" ]
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: alpine
kubeclarity logs show:
time="2022-04-27T16:28:57Z" level=info msg="Start scanning..." func="github.com/cisco-open/kubei/runtime_scan/pkg/scanner.(*Scanner).Scan" file="/build/runtime_scan/pkg/scanner/scanner.go:265" scanner id=531f4af4-60e3-4077-a0b1-5984b862e080
time="2022-04-27T16:28:57Z" level=debug msg="worker #1 halted" func="github.com/cisco-open/kubei/runtime_scan/pkg/scanner.(*Scanner).worker" file="/build/runtime_scan/pkg/scanner/job_managment.go:128" scanner id=f880455f-e909-4fa5-8d7f-2c38a92d66c2
time="2022-04-27T16:28:57Z" level=debug msg="worker #6 halted" func="github.com/cisco-open/kubei/runtime_scan/pkg/scanner.(*Scanner).worker" file="/build/runtime_scan/pkg/scanner/job_managment.go:128" scanner id=f880455f-e909-4fa5-8d7f-2c38a92d66c2
time="2022-04-27T16:28:57Z" level=debug msg="worker #8 halted" func="github.com/cisco-open/kubei/runtime_scan/pkg/scanner.(*Scanner).worker" file="/build/runtime_scan/pkg/scanner/job_managment.go:128" scanner id=f880455f-e909-4fa5-8d7f-2c38a92d66c2
time="2022-04-27T16:28:57Z" level=debug msg="worker #7 halted" func="github.com/cisco-open/kubei/runtime_scan/pkg/scanner.(*Scanner).worker" file="/build/runtime_scan/pkg/scanner/job_managment.go:128" scanner id=f880455f-e909-4fa5-8d7f-2c38a92d66c2
time="2022-04-27T16:28:57Z" level=debug msg="worker #9 halted" func="github.com/cisco-open/kubei/runtime_scan/pkg/scanner.(*Scanner).worker" file="/build/runtime_scan/pkg/scanner/job_managment.go:128" scanner id=f880455f-e909-4fa5-8d7f-2c38a92d66c2
time="2022-04-27T16:28:57Z" level=debug msg="worker #2 halted" func="github.com/cisco-open/kubei/runtime_scan/pkg/scanner.(*Scanner).worker" file="/build/runtime_scan/pkg/scanner/job_managment.go:128" scanner id=f880455f-e909-4fa5-8d7f-2c38a92d66c2
time="2022-04-27T16:28:57Z" level=debug msg="worker #5 halted" func="github.com/cisco-open/kubei/runtime_scan/pkg/scanner.(*Scanner).worker" file="/build/runtime_scan/pkg/scanner/job_managment.go:128" scanner id=f880455f-e909-4fa5-8d7f-2c38a92d66c2
time="2022-04-27T16:28:57Z" level=debug msg="worker #4 halted" func="github.com/cisco-open/kubei/runtime_scan/pkg/scanner.(*Scanner).worker" file="/build/runtime_scan/pkg/scanner/job_managment.go:128" scanner id=f880455f-e909-4fa5-8d7f-2c38a92d66c2
time="2022-04-27T16:28:57Z" level=debug msg="worker #3 halted" func="github.com/cisco-open/kubei/runtime_scan/pkg/scanner.(*Scanner).worker" file="/build/runtime_scan/pkg/scanner/job_managment.go:128" scanner id=f880455f-e909-4fa5-8d7f-2c38a92d66c2
time="2022-04-27T16:28:57Z" level=debug msg="No matching secret found." func=github.com/cisco-open/kubei/shared/pkg/utils/k8s.GetMatchingSecretName file="/build/shared/pkg/utils/k8s/k8s_utils.go:71"
time="2022-04-27T16:28:57Z" level=info msg="Total 1 unique images to scan" func="github.com/cisco-open/kubei/runtime_scan/pkg/scanner.(*Scanner).initScan" file="/build/runtime_scan/pkg/scanner/scanner.go:255" scanner id=531f4af4-60e3-4077-a0b1-5984b862e080
time="2022-04-27T16:28:57Z" level=debug msg="Created job=&Job{ObjectMeta:{scanner-alpinetest-1b21b6dc-1e25-45fa-a94f-5993c5109577 mattzap 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[app:scanner sidecar.istio.io/inject:false] map[] [] [] []},Spec:JobSpec{Parallelism:nil,Completions:nil,ActiveDeadlineSeconds:nil,Selector:nil,ManualSelector:nil,Template:{{ 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[app:scanner sidecar.istio.io/inject:false] map[] [] [] []} {[{tmp-volume {nil EmptyDirVolumeSource{Medium:,SizeLimit:<nil>,} nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil}}] [] [{vulnerability-scanner ghcr.io/cisco-open/kubeclarity-runtime-k8s-scanner:latest [] [scan --log-level debug] [] [] [{RESULT_SERVICE_ADDR kubeclarity-kubeclarity.kubeclarity:8888 nil} {SBOM_DB_ADDR kubeclarity-kubeclarity-sbom-db.kubeclarity:8080 nil} {ANALYZER_LIST syft gomod nil} {ANALYZER_SCOPE squashed nil} {SCANNERS_LIST grype nil} {SCANNER_GRYPE_MODE REMOTE nil} {REGISTRY_SKIP_VERIFY_TLS false nil} {REGISTRY_USE_HTTP false nil} {SCANNER_REMOTE_GRYPE_SERVER_ADDRESS kubeclarity-kubeclarity-grype-server.kubeclarity:9991 nil} {SCANNER_REMOTE_GRYPE_SERVER_TIMEOUT 2m nil} {SCAN_UUID a3033112-a35d-4574-8532-4112bec76318 nil} {IMAGE_ID_TO_SCAN uk.icr.io/sbu-pipeline/alpinetest@sha256:213d60a32697894f9a1dbe3f309a5b5a737266a1b7866594937038514ebdc577 nil} {IMAGE_HASH_TO_SCAN 213d60a32697894f9a1dbe3f309a5b5a737266a1b7866594937038514ebdc577 nil} {IMAGE_NAME_TO_SCAN uk.icr.io/sbu-pipeline/alpinetest:3.14.3 nil}] {map[cpu:{{1000 -3} {<nil>} DecimalSI} memory:{{1048576000 0} {<nil>} BinarySI}] map[cpu:{{50 -3} {<nil>} 50m DecimalSI} memory:{{52428800 0} {<nil>} 50Mi BinarySI}]} [{tmp-volume false /tmp <nil> }] [] nil nil nil nil Always &SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[all],},Privileged:*false,SELinuxOptions:nil,RunAsUser:nil,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,} false false false}] [] Never <nil> <nil> map[] <nil> false false false <nil> nil [] nil [] [] <nil> nil [] <nil> <nil> <nil> map[] [] <nil> nil}},BackoffLimit:*0,TTLSecondsAfterFinished:*300,CompletionMode:nil,Suspend:nil,},Status:JobStatus{Conditions:[]JobCondition{},StartTime:<nil>,CompletionTime:<nil>,Active:0,Succeeded:0,Failed:0,CompletedIndexes:,UncountedTerminatedPods:nil,Ready:nil,},}" func="github.com/cisco-open/kubei/runtime_scan/pkg/scanner.(*Scanner).runJob" file="/build/runtime_scan/pkg/scanner/job_managment.go:164" scanner id=531f4af4-60e3-4077-a0b1-5984b862e080
time="2022-04-27T16:28:57Z" level=info msg="Running job mattzap/scanner-alpinetest-1b21b6dc-1e25-45fa-a94f-5993c5109577 to scan imageID uk.icr.io/sbu-pipeline/alpinetest@sha256:213d60a32697894f9a1dbe3f309a5b5a737266a1b7866594937038514ebdc577" func="github.com/cisco-open/kubei/runtime_scan/pkg/scanner.(*Scanner).runJob" file="/build/runtime_scan/pkg/scanner/job_managment.go:166" scanner id=531f4af4-60e3-4077-a0b1-5984b862e080
time="2022-04-27T16:28:57Z" level=info msg="Waiting for result. imageID=uk.icr.io/sbu-pipeline/alpinetest@sha256:213d60a32697894f9a1dbe3f309a5b5a737266a1b7866594937038514ebdc577" func="github.com/cisco-open/kubei/runtime_scan/pkg/scanner.(*Scanner).waitForResult" file="/build/runtime_scan/pkg/scanner/job_managment.go:135" scanner id=531f4af4-60e3-4077-a0b1-5984b862e080
time="2022-04-27T16:29:06Z" level=info msg="Vulnerabilities result was set for imageID \"uk.icr.io/sbu-pipeline/alpinetest@sha256:213d60a32697894f9a1dbe3f309a5b5a737266a1b7866594937038514ebdc577\"" func="github.com/cisco-open/kubei/runtime_scan/pkg/scanner.(*Scanner).HandleScanResults" file="/build/runtime_scan/pkg/scanner/scanner.go:392" scanner id=531f4af4-60e3-4077-a0b1-5984b862e080
time="2022-04-27T16:29:06Z" level=warning msg="Vulnerabilities scan of imageID \"uk.icr.io/sbu-pipeline/alpinetest@sha256:213d60a32697894f9a1dbe3f309a5b5a737266a1b7866594937038514ebdc577\" has failed: &{failed to analyze image: failed to run job manager: failed to run job: failed to create source analyzer=syft: could not fetch image 'uk.icr.io/sbu-pipeline/alpinetest@sha256:213d60a32697894f9a1dbe3f309a5b5a737266a1b7866594937038514ebdc577': unable determine image source TBD}" func="github.com/cisco-open/kubei/runtime_scan/pkg/scanner.(*Scanner).HandleScanResults" file="/build/runtime_scan/pkg/scanner/scanner.go:399" scanner id=531f4af4-60e3-4077-a0b1-5984b862e080
time="2022-04-27T16:29:06Z" level=info msg="Image scanned result has arrived. imageID=uk.icr.io/sbu-pipeline/alpinetest@sha256:213d60a32697894f9a1dbe3f309a5b5a737266a1b7866594937038514ebdc577" func="github.com/cisco-open/kubei/runtime_scan/pkg/scanner.(*Scanner).waitForResult" file="/build/runtime_scan/pkg/scanner/job_managment.go:139" scanner id=531f4af4-60e3-4077-a0b1-5984b862e080
time="2022-04-27T16:29:06Z" level=info msg="All jobs has finished" func="github.com/cisco-open/kubei/runtime_scan/pkg/scanner.(*Scanner).jobBatchManagement" file="/build/runtime_scan/pkg/scanner/job_managment.go:93" scanner id=531f4af4-60e3-4077-a0b1-5984b862e080
time="2022-04-27T16:29:06Z" level=info msg="Succeeded to apply runtime scan results. app ids=[], failures=[Failed to scan image \"uk.icr.io/sbu-pipeline/alpinetest:3.14.3\".\nEffected pods: alpine-74d9957b8d-gn4ct/mattzap.\nReasons: failed to analyze image: failed to run job manager: failed to run job: failed to create source analyzer=syft: could not fetch image 'uk.icr.io/sbu-pipeline/alpinetest@sha256:213d60a32697894f9a1dbe3f309a5b5a737266a1b7866594937038514ebdc577': unable determine image source.]" func="github.com/cisco-open/kubei/backend/pkg/rest.(*Server).startScan.func1" file="/build/backend/pkg/rest/runtime_scan_controller.go:255"
Scanner logs show (had to grab from a subsequent run as a I forgot):
$ oc logs scanner-alpinetest-1898289c-def5-4726-88ab-ad614ba7549--1-x48bk
time="2022-04-27T16:42:26Z" level=debug msg="Credentials not found. image name=uk.icr.io/sbu-pipeline/alpinetest@sha256:213d60a32697894f9a1dbe3f309a5b5a737266a1b7866594937038514ebdc577." func="github.com/cisco-open/kubei/shared/pkg/utils/creds.(*CredExtractor).GetCredentials" file="/build/shared/pkg/utils/creds/extractor.go:78"
time="2022-04-27T16:42:26Z" level=info msg="\n\nconfig={\"ResultServiceAddress\":\"kubeclarity-kubeclarity.kubeclarity:8888\",\"SBOMDBAddress\":\"kubeclarity-kubeclarity-sbom-db.kubeclarity:8080\",\"ImageIDToScan\":\"uk.icr.io/sbu-pipeline/alpinetest@sha256:213d60a32697894f9a1dbe3f309a5b5a737266a1b7866594937038514ebdc577\",\"ImageHashToScan\":\"213d60a32697894f9a1dbe3f309a5b5a737266a1b7866594937038514ebdc577\",\"ScanUUID\":\"c7d0e605-31bd-4073-8075-e85ab9d2845d\",\"RegistryInsecure\":false,\"SharedConfig\":{\"Registry\":{\"skip-verify-tls\":false,\"use-http\":false,\"auths\":[{\"authority\":\"\"}]},\"Analyzer\":{\"OutputFormat\":\"cyclonedx\",\"AnalyzerList\":[\"syft\",\"gomod\"],\"Scope\":\"squashed\"},\"Scanner\":{\"ScannersList\":[\"grype\"],\"GrypeConfig\":{\"UpdateDB\":true,\"DBRootDir\":\"/tmp/\",\"ListingURL\":\"https://toolbox-data.anchore.io/grype/databases/listing.json\",\"Scope\":\"Squashed\",\"GrypeServerAddress\":\"kubeclarity-kubeclarity-grype-server.kubeclarity:9991\",\"GrypeServerTimeout\":120000000000,\"Mode\":\"REMOTE\"},\"DependencyTrackConfig\":{\"host\":\"dependency-track-apiserver.dependency-track\",\"project-name\":\"\",\"project-version\":\"\",\"should-delete-project\":true,\"disable-tls\":false,\"insecure-skip-verify\":true,\"fetch-vulnerabilities-retry-count\":5,\"fetch-vulnerabilities-retry-sleep\":30000000000}},\"LocalImageScan\":false},\"ImageNameToScan\":\"uk.icr.io/sbu-pipeline/alpinetest:3.14.3\"}\n\n" func=github.com/cisco-open/kubei/runtime_k8s_scanner/pkg/config.LoadConfig file="/build/runtime_k8s_scanner/pkg/config/config.go:61"
time="2022-04-27T16:42:26Z" level=info msg="Runtime K8s Scanner job is running. imageID=uk.icr.io/sbu-pipeline/alpinetest@sha256:213d60a32697894f9a1dbe3f309a5b5a737266a1b7866594937038514ebdc577, hash=213d60a32697894f9a1dbe3f309a5b5a737266a1b7866594937038514ebdc577" func=github.com/cisco-open/kubei/runtime_k8s_scanner/pkg/scanner.Run file="/build/runtime_k8s_scanner/pkg/scanner/scanner.go:51"
time="2022-04-27T16:42:26Z" level=info msg="Called syft analyzer on source registry:uk.icr.io/sbu-pipeline/alpinetest@sha256:213d60a32697894f9a1dbe3f309a5b5a737266a1b7866594937038514ebdc577" analyzer=syft component=analyzer image-id="uk.icr.io/sbu-pipeline/alpinetest@sha256:213d60a32697894f9a1dbe3f309a5b5a737266a1b7866594937038514ebdc577" scan-uuid=c7d0e605-31bd-4073-8075-e85ab9d2845d
time="2022-04-27T16:42:26Z" level=error msg="failed to analyze image: failed to run job manager: failed to run job: failed to create source analyzer=syft: could not fetch image 'uk.icr.io/sbu-pipeline/alpinetest@sha256:213d60a32697894f9a1dbe3f309a5b5a737266a1b7866594937038514ebdc577': unable determine image source" image-id="uk.icr.io/sbu-pipeline/alpinetest@sha256:213d60a32697894f9a1dbe3f309a5b5a737266a1b7866594937038514ebdc577" scan-uuid=c7d0e605-31bd-4073-8075-e85ab9d2845d
time="2022-04-27T16:42:26Z" level=info msg="Runtime K8s Scanner job was done." func=github.com/cisco-open/kubei/runtime_k8s_scanner/pkg/scanner.Run file="/build/runtime_k8s_scanner/pkg/scanner/scanner.go:64"
from kubeclarity.
@mtcolman, thank you for the detailed information.
There are several checks that are being done for the image pull secret to make sure that it can be used:
https://github.com/cisco-open/kubei/blob/ea46fc3cbe0dc819e57c18511dc0c96bb24e48b2/shared/pkg/utils/k8s/k8s_utils.go#L49
From the code, I see that if the password and the username are both empty, the image pull secret is being ignore, disregarding the additional Auth property.
Can you please check if that's indeed the case in the sbu-pipeline secret.
There are some additional checks that the auth corresponds to the image, that might also cause the mismatch, but i'm doubted as K8s manages to pull the image and it uses the same pull secret code as we do.
The matching logic is tested in this unit test:
https://github.com/cisco-open/kubei/blob/ea46fc3cbe0dc819e57c18511dc0c96bb24e48b2/shared/pkg/utils/k8s/k8s_utils_test.go#L65
maybe you can add your specific structure to the unit test to see that it's matched. Also, we can test it if you can strip all sensitive information (e.g. replace with XXX) and share the secret json.
Thanks!!!
from kubeclarity.
@akpsgit the username and password are both supplied in sbu-pipeline
and this is validated by the fact the image can be pulled when the pod is deployed. - this is shown in the below comment as well for evidence.
from kubeclarity.
Here is the secret:
$ oc get secret sbu-pipeline -n mattzap -oyaml
apiVersion: v1
data:
.dockerconfigjson: <base64-redacted-by-me>
kind: Secret
metadata:
annotations:
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"v1","data":{".dockerconfigjson":"<base64-redacted-by-me>"},"kind":"Secret","metadata":{"annotations":{},"creationTimestamp":null,"name":"sbu-pipeline","namespace":"mattzap"},"type":"kubernetes.io/dockerconfigjson"}
creationTimestamp: "2022-04-27T16:05:45Z"
name: sbu-pipeline
namespace: mattzap
resourceVersion: "706789537"
uid: da8364a4-d023-4281-a2a9-22b2b88bfaac
type: kubernetes.io/dockerconfigjson
Here is me showing the decoded base64:
$ echo -n <base64-redacted-by-me> | base64 -d
{"auths":{"uk.icr.io/sbu-pipeline":{"username":"iamapikey","password":"<redacted-by-me>","email":"unused","auth":"<base64-encoded-iamapikey:password>"}}}
from kubeclarity.
@mmurhamm sorry for the inconvenience, it seems like that if you will use registry name uk.icr.io
or the full target name uk.icr.io/sbu-pipeline/alpinetest
in the image pull secret it will work.
We need to dig more to understand what was changes between the old logic and the new one.
{"auths":{"uk.icr.io":{"username":"iamapikey","password":"<redacted-by-me>","email":"unused","auth":"<base64-encoded-iamapikey:password>"}}}
OR
{"auths":{"uk.icr.io/sbu-pipeline/alpinetest":{"username":"iamapikey","password":"<redacted-by-me>","email":"unused","auth":"<base64-encoded-iamapikey:password>"}}}
from kubeclarity.
Hi @mtcolman v2.2.0 just released, can we close the issue?
from kubeclarity.
Related Issues (20)
- Failed to Scan a Image but the Image exists in the repository HOT 5
- [Feature]: Columns to flag and filter CVEs in order to hide from the UI HOT 2
- Include a possibility to exclude some PODS in Scanner via UI or YAML by Label HOT 2
- Failed to scan image : unable to determine image source to select platform HOT 10
- KubeClarity Runtime scan fails | SBOM not generated when only Trivy configured as the analyzers cum scanner HOT 12
- certifi dependency false positive. HOT 3
- clone(openclarity#483): Option to add pathPrefix to kubeclarity
- clone(openclarity#485): Add ADOPTERS.md to KubeClarity
- clone(openclarity#486): Support for trivy token in helm charts
- clone(openclarity#488): Unable to analyze/scan java images with Trivy
- clone(openclarity#498): Failed to analyze local images with trivy
- clone(openclarity#507): E2E tests should cover kubeclarity-cli analyze then kubeclarity-cli scan
- clone(openclarity#511): unable to use OciRegistry source error for runtime scanning
- clone(openclarity#515): Cant run scan on openshift - Job creation fail
- clone(openclarity#532): Which command name is correct? `kubeclarity` or `kubeclarity-cli`?
- clone(openclarity#545): New release (to get DB SSL mode)
- clone(openclarity#555): Consider removing or making the `helm.sh/hook: pre-install` annotations optional
- Exporting scan result, vulnerable, overview in PDF, Excel, CSV file HOT 6
- clone(openclarity#566): Add a method in the UI to allow or ignore vulnerabilities
- clone(openclarity#567): Make it easier for users to correlate an application to specific pod(s) in the cluster
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from kubeclarity.