quay / claircore Goto Github PK

View Code? Open in Web Editor NEW

132.0 19.0 80.0 41.27 MB

foundation modules for scanning container packages and reporting vulnerabilities

Home Page: https://quay.github.io/claircore/

License: Apache License 2.0

Makefile 0.11% Go 99.31% Dockerfile 0.04% PLpgSQL 0.22% Shell 0.30% Handlebars 0.01% jq 0.02%

golang hacktoberfest static-analysis

claircore's Introduction

ClairCore

ClairCore provides a set of go modules which handle scanning container layers for installed packages and reporting any discovered vulnerabilities.
ClairCore is designed to be embedded into a service wrapper.

For a full overview see: ClairCore Book

Local development and testing

The following targets start and stop a local development environment

make local-dev-up
make local-dev-down

If you modify libvuln or libindex code the following make targets will restart the services with your changes

make libindexhttp-restart
make libvulnhttp-restart

With the local development environment up the following make target runs all tests including integration

make integration

The following make target runs unit tests which do not require a database or local development environment

make unit

claircore's People

Contributors

Stargazers

Watchers

Forkers

allda jpkrohling exdx pombredanne hdonnay greut pedrofarinhaatshiftleftpt alecmerdler ppadmavilasom arajkumar priyanka19-98 rafiu007 laashub-soa ashwin-h jonathankingfc jasinner jan-zmeskal sofyatavrovskaya liubogithub westonsteimel syed ldelossa clix-dev-llc crozzy jvulgan armosec majewsky x-ops panzer1119 noseka1 yzainee-zz odidev harishsurf nanikjava siwinski neijurli gnaponie misberner kairen xxpolygonxx paulaldridge ammachado ekmixon bluefriday iainduncani isabella232 robertocodeeye tonyfieit alexkoltun toob12 4math2379 bored-engineer mayurwaghmode elicharlese kripanshdevtron rtann astebenkova slumericanbx daynewlee ksdeekshith raja-0940 dfodorrh ernesgonzalez33 ruromero purplexed vishnuchalla stewartsmith dims frostmar jbergstroem bradlugo celek pandatix jvdm stackrox kieran-muller janisz maddosaurus ludydoo phmarek

claircore's Issues

runtime.NumCPU possibly unreliable in containers

I think that runtime.NumCPU() will report the number of cores on the machine (ala nproc(1)) which doesn't represent the number of cores actually available. If this is the case, some of our queue sizing will be too large when containerized.

Plumb dist.DID through data layer

Currently Distribution.DID is in not plumbed from the package scanner to the data model. We currently do not record DID.

Support vulnerability listing for maven central repo artifacts

Maven is a package repository for Java.

The mvn tool which packs the maven projects usually adds metadata fields(dependency name, version) into META-INF/maven/<groupId>/<artifactId> which could be found on every jar file.

There are two files with in every maven jar which would give us the necessary information like package|version,

META-INF/maven///pom.properties
META-INF/maven///pom.xml

The pom.xml and pom.properties files are packaged up in the JAR so that each artifact produced by Maven is self-describing and also allows you to utilize the metadata in your own application if the need arises. One simple use might be to retrieve the version of your application. Operating on the POM file would require you to use some Maven utilities but the properties can be utilized using the standard Java API and look like the following:

#Generated by Maven
#Tue Oct 04 15:43:21 GMT-05:00 2005
version=1.0-SNAPSHOT
groupId=com.mycompany.app
artifactId=my-app

Reference: https://maven.apache.org/guides/getting-started/#how-do-i-add-resources-to-my-jar

[Question] Connection between Clair and ClairCore

Hi, I am familiar with Clair and I am wondering what is the connection between Clair and ClairCore?

Is this a library that will be used in Clair in the future?

Vulnerability Severity Should be an Enum

Description

In previous Clair implementation, the Severity of a given Vulnerability would be one of a defined enum. Currently, this struct field is typed as string. We can make this a static type.

check for gnu-tar on osx

Default installation of tar on MacOS will return errors when the tar tool is used. This issue is exposed in the RPM package scanner.

[updater] investigate streaming sha + writing oval databases to disk

Currently we read the oval xml databases into memory to get a sha256 hash. It would be more efficient to take a streaming sha256 of the contents while writing to disk.

Resources will need to be purged once the updater has finished parsing the xml database.

Clair v4 and image layers with mediaType ".tar.gzip" are unknown content-type

Description

Hi team :D
I am migrating a project using Clair v3 to use Clair v4.
I am getting an error(failed to fetch layers: encountered error while fetching a layer: fetcher: unknown content-type "application/vnd.docker.image.rootfs.diff.tar.gzip) after I run a scan of an image stored in the nexus repo.

After digging I think I found the root of the error I got here:

claircore/internal/indexer/fetcher/fetcher.go

Lines 181 to 206 in 3a4e3d3

    
           switch { 
        
           case ct == "application/gzip": 
        
           	fallthrough 
        
           case strings.HasSuffix(ct, ".tar+gzip"): 
        
           	g, err := gzip.NewReader(br) 
        
           	if err != nil { 
        
           		return err 
        
           	} 
        
           	defer g.Close() 
        
           	r = g 
        
           case ct == "application/zstd": 
        
           	fallthrough 
        
           case strings.HasSuffix(ct, ".tar+zstd"): 
        
           	s, err := zstd.NewReader(br) 
        
           	if err != nil { 
        
           		return err 
        
           	} 
        
           	defer s.Close() 
        
           	r = s 
        
           case ct == "application/x-tar": 
        
           	fallthrough 
        
           case strings.HasSuffix(ct, ".tar"): 
        
           	r = br 
        
           default: 
        
           	return fmt.Errorf("fetcher: unknown content-type %q", ct) 
        
           }

Seems the format "application / vnd.docker.image.rootfs.diff.tar.gzip" is not found in the rules, goes to "default" and return the error

Payload and responses

Image manifest

{
   "schemaVersion": 2,
   "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
   "config": {
      "mediaType": "application/vnd.docker.container.image.v1+json",
      "size": 10077,
      "digest": "sha256:4741a49ceb16bb4a0cf5c6afc259f9a9a437a3bb7cdb15d38a500076c6d43611"
   },
   "layers": [
      {
         "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
         "size": 45344292,
         "digest": "sha256:741437d97401b83849ccbfe4ed8964049b752081647d7f1ea8aea29d989f8968"
      },
      {
         "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
         "size": 10781099,
         "digest": "sha256:34d8874714d74b636739b8a52204650a664fca8ff9741dd66810f30196f103e2"
      },....
   ]
}

Payload to Clair API

{
  "hash": "sha256:e1a7cf003f4b14a70bc057b31a7bc7823663c041e7cbdd8ed99d59de09573b3a",
  "layers": [
    {
      "hash": "sha256:741437d97401b83849ccbfe4ed8964049b752081647d7f1ea8aea29d989f8968",
      "uri": "https://<nexus-repositoty>/v2/<image>/blobs/sha256:741437d97401b83849ccbfe4ed8964049b752081647d7f1ea8aea29d989f8968",
      "headers": {
        "Authorization": [
          "Bearer <token>"
        ]
      }
    }
  ]
}

Stderr

	"level": "error",
	"component": "init/NewHttpTransport",
	"component": "libindex/Libindex.Index",
	"manifest": "sha256:e1a7cf003f4b14a70bc057b31a7bc7823663c041e7cbdd8ed99d59de09573b3a",
	"component": "libindex/Libindex.index",
	"component": "internal/indexer/controller/Controller.Index",
	"manifest": "sha256:e1a7cf003f4b14a70bc057b31a7bc7823663c041e7cbdd8ed99d59de09573b3a",
	"state": "CheckManifest",
	"error": "failed to fetch layers: encountered error while fetching a layer: fetcher: unknown content-type \"application/vnd.docker.image.rootfs.diff.tar.gzip\"",
	"time": "2021-01-28T17:41:53Z",
	"time": "2021-01-28T17:41:53Z",
	"message": "error during scan"
}

Reponse from Clair API

    "manifest_hash": "sha256:e1a7cf003f4b14a70bc057b31a7bc7823663c041e7cbdd8ed99d59de09573b3a",
    "state": "IndexError",
    "packages": {},
    "distributions": {},
    "repository": {},
    "environments": {},
    "success": false,
    "err": "failed to fetch layers: encountered error while fetching a layer: fetcher: unknown content-type \"application/vnd.docker.image.rootfs.diff.tar.gzip\""
}

Cheers
Ederson :)

Remove nameversion field from Package

Remove the nameversion field from the claircore.Package struct. It is not used by any functionality.

Linux coalescer doesn't remove obsolete packages

Linux coalescer includes packages that are no present in the final container image into the index report. This happens when a package is installed in a layer and later either updated, downgraded or removed in other layers. Current coalescer implementation keeps both (original and updated) in index report even though original package is no longer in an image.

Here is an example of such a index report with updated package.

{
  "packages": {
    "4040": {
      "id": "4040",
      "name": "vim-minimal",
      "version": "2:7.4.160-3.el7",
      "kind": "binary",
      "source": {
        "id": "4039",
        "name": "vim",
        "version": "7.4.160-3.el7",
        "kind": "source",
        "normalized_version": "",
        "cpe": ""
      },
      "normalized_version": "",
      "arch": "x86_64",
      "cpe": ""
    },

    "4324": {
      "id": "4324",
      "name": "vim-minimal",
      "version": "2:7.4.160-4.el7",
      "kind": "binary",
      "source": {
        "id": "4323",
        "name": "vim",
        "version": "7.4.160-4.el7",
        "kind": "source",
        "normalized_version": "",
        "cpe": ""
      },
      "normalized_version": "",
      "arch": "x86_64",
      "cpe": ""
    }
  }
}

The package vim-minimal-2:7.4.160-3.el7 is not present in final image because it was replaced by vim-minimal-2:7.4.160-4.el7

alpine 3.7 parser error

libvuln reports a parsing error for alpine 3.4

ERR error from updater: updater alpine-main-v3.4-updater failed to update: failed to parse the fetched vulnerability database: yaml: unmarshal errors:
  line 37: mapping key "2.4.27-r1" already defined at line 27 component=libvuln

fix to sec db has been opened:
alpinelinux/alpine-secdb#5

Waiting for comments or merger

Request Tracing

If this is going to become a part of the Quay stack, we'll want to make sure it's compatible with the distributed tracing solution.

cc @jpkrohling

add "display" name to updaters

Since API users are surfacing updater names to humans (see quay/quay#937), we should consider adding a nicer looking name to updaters for this purpose.

The current updater name is just a system-unique string used as an ID for various updater processes.
We could also develop a heuristic to generate a nicer looking name for progressive enhancement.

Unable to leverage advanced connstring arguments in indexer

When I was reading the code of pgxpool library, I realized that user should be able to enrich PostgreSQL connstring by advanced arguments that are parsed by pgxpool. See the ParseConfig function. However, if you actually try to do that, you'll get this:

I was running indexer with these settings:

indexer:
  connstring: host=<censored> port=5432 dbname=<censored> user=<censored> password=<censored> sslmode=disable application_name=clair-indexer pool_max_conns=1 pool_max_conn_idle_time=5m0s
  scanlock_retry: 10
  layer_scan_concurrency: 5
  migrations: True

Workaround is to set migrations to False. However, long-term, I need to run indexer with migrations set to True.

Clair v4.0 doesn't find all installed packages in debian- and ubuntu-based images

Clair v4.0 doesn't find all installed packages in debian- and ubuntu-based images:

For example, Clair v4.0 doesn't find following packages in debian-based nginx:1.18 image:

nginx 1.18.0-2~buster
nginx-module-image-filter 1.18.0-2~buster
nginx-module-geoip 1.18.0-2~buster
nginx-module-njs 1.18.0.0.4.4-2~buster
nginx-module-xslt 1.18.0-2~buster
ca-certificates 20200601~deb10u1
curl 7.64.0-4+deb10u1
openssl 1.1.1d-0+deb10u4

But this packages are inside image nginx:1.18:

root@80a04257c5ba:/# dpkg -l | grep nginx  
ii  nginx                     1.18.0-2~buster             amd64        high performance web server
ii  nginx-module-geoip        1.18.0-2~buster             amd64        nginx GeoIP dynamic modules
ii  nginx-module-image-filter 1.18.0-2~buster             amd64        nginx image filter dynamic module
ii  nginx-module-njs          1.18.0.0.4.4-2~buster       amd64        nginx njs dynamic modules
ii  nginx-module-xslt         1.18.0-2~buster             amd64        nginx xslt dynamic module

root@80a04257c5ba:/# dpkg -l | grep openssl
ii  openssl                   1.1.1d-0+deb10u4            amd64        Secure Sockets Layer toolkit - cryptographic utility

root@80a04257c5ba:/# dpkg -l | grep curl 
ii  curl                      7.64.0-4+deb10u1            amd64        command line tool for transferring data with URL syntax

root@80a04257c5ba:/# dpkg -l | grep openssl
ii  openssl                   1.1.1d-0+deb10u4            amd64        Secure Sockets Layer toolkit - cryptographic utility

The similar problem is observed in taken for example ubuntu-based itzg/openssl-ubuntu:latest image. Clair v4.0 can't detect following packages:

openssl 1.1.1-1ubuntu2.1~18.04.4
libssl1.1 1.1.1-1ubuntu2.1~18.04.4

which is installed inside image itzg/openssl-ubuntu:latest :

ii  openssl                 1.1.1-1ubuntu2.1~18.04.4 amd64        Secure Sockets Layer toolkit - cryptographic utility

root@4459c2b830e4:/certs# dpkg -l | grep libssl1.1
ii  libssl1.1:amd64         1.1.1-1ubuntu2.1~18.04.4 amd64        Secure Sockets Layer toolkit - shared libraries

It's look like regression, because Clair v2.0 detects these packages.

JSON-reports for nginx:1.18 and itzg/openssl-ubuntu:latest from Clair v4.0 in attachments.
openssl-ubuntu.txt
nginx.txt

CentOS Support

When inventorying issues in Clair I discovered a CentOS vulnerability source PR introduced by @irenexychen
quay/clair#761

Looks like CentOS has a security database called CESA. I was unaware of this. We should begin investigating CentOS support and this security database.

Incorrect version parsing from dpkg archive

Description of Problem / Feature Request

JSON report after scanning image nginx:1.18 has incorrect version entry:

198":{"id":"198","name":"debian-archive-keyring","version":"2019.1 /etc/apt/trusted.gpg.d/debian-archive-buster-automatic.gpg 9e93d0a43d3a60272034c15900e9df6f /etc/apt/trusted.gpg.d/debian-archive-buster-security-automatic.gpg f2d1b03b7a3c279ec66425d06aaab50f /etc/apt/trusted.gpg.d/debian-archive-buster-stable.gpg 4797ff6df738da65413ef710cf73936f /etc/apt/trusted.gpg.d/debian-archive-jessie-automatic.gpg 47d3fff11215d63917b41cb249ca0cbb /etc/apt/trusted.gpg.d/debian-archive-jessie-security-automatic.gpg 762c194d687970dd37e6bbcb1f88be6b /etc/apt/trusted.gpg.d/debian-archive-jessie-stable.gpg 396bc7a1b3a1c2a67b33366b9300897b /etc/apt/trusted.gpg.d/debian-archive-stretch-automatic.gpg f8ca9f176f6a5747e113f62220671e0b /etc/apt/trusted.gpg.d/debian-archive-stretch-security-automatic.gpg 986449e3c1ed5c157686f0166411b829 /etc/apt/trusted.gpg.d/debian-archive-stretch-stable.gpg 67fa5396fa0900c0abd1058d98d9247e","kind":"binary","source":{"id":"33","name":"","version":"","normalized_version":"","cpe":""},"normalized_version":"","arch":"all","cpe":""}

Expected result:
198":{"id":"198","name":"debian-archive-keyring","version":"2019.1","kind":"binary","source":{"id":"33","name":"","version":"","normalized_version":"","cpe":""},"normalized_version":"","arch":"all","cpe":""}

It seems that Clair v4.0 parse '/var/lib/dpkg/status' file incorrectly, because it contains following entry:

Status: install ok installed
Priority: important
Section: misc
Installed-Size: 198
Maintainer: Debian Release Team <[email protected]>
Architecture: all
Multi-Arch: foreign
Version: 2019.1
Conffiles:
 /etc/apt/trusted.gpg.d/debian-archive-buster-automatic.gpg 9e93d0a43d3a60272034c15900e9df6f
 /etc/apt/trusted.gpg.d/debian-archive-buster-security-automatic.gpg f2d1b03b7a3c279ec66425d06aaab50f
 /etc/apt/trusted.gpg.d/debian-archive-buster-stable.gpg 4797ff6df738da65413ef710cf73936f
 /etc/apt/trusted.gpg.d/debian-archive-jessie-automatic.gpg 47d3fff11215d63917b41cb249ca0cbb
 /etc/apt/trusted.gpg.d/debian-archive-jessie-security-automatic.gpg 762c194d687970dd37e6bbcb1f88be6b
 /etc/apt/trusted.gpg.d/debian-archive-jessie-stable.gpg 396bc7a1b3a1c2a67b33366b9300897b
 /etc/apt/trusted.gpg.d/debian-archive-stretch-automatic.gpg f8ca9f176f6a5747e113f62220671e0b
 /etc/apt/trusted.gpg.d/debian-archive-stretch-security-automatic.gpg 986449e3c1ed5c157686f0166411b829
 /etc/apt/trusted.gpg.d/debian-archive-stretch-stable.gpg 67fa5396fa0900c0abd1058d98d9247e
Description: GnuPG archive keys of the Debian archive
 The Debian project digitally signs its Release files. This package
 contains the archive keys used for that.

Fetcher needs to check CTX

Found out in #149 that fetcher does not check it's ctx for cancellation at all.
Refactor fetcher to handle ctx cancellation gracefully.

Export interfaces used in factory methods

Any interfaces used in factory methods such as updater.Updater and matcher.Matcher must be exported and moved out of the /internal package. Keeping these interfaces in the internal package means the function signature cannot be created in someone else's codebase.

updater: failed to parse alpine-main-v3.6 and alpine-community-v3.3 JSON vulnerability databases

Alpine-updater fails when trying parse alpine-main-v3.6 and alpine-community-v3.3 JSON vulnerability database.

Clair-matcher logs:
ERR encountered error while updating error="updating errors:\n\talpine-main-v3.6-updater: failed to parse the fetched vulnerability database: json: cannot unmarshal object into Go struct field Details.packages.pkg.secfixes of type []string\n\talpine-community-v3.3-updater: failed to parse the fetched vulnerability database: json: cannot unmarshal object into Go struct field SecurityDB.packages of type []alpine.Package\n" component=libvuln/New

Clair version/image: clairV4.0.
Claircore: v0.1.20

INT package and distribution ID may not be large enough

Currently package and distribution structs use INT for their ID. These IDs uniquely identify the given resource. There's some concern about the number of unique packages overflowing an int's space.

Solutions:
Use UUID in postgres as the primary key. pgcrypto extensions is necessary
Generate unique name derived from fields on the entity such as "name"
Big serial can be used as the primary key in the database

I am leaning toward UUID in the database. @hdonnay @jzelinskie curious about opinions.

Add a Golang core scanner to ClairCore

Description Feature Request (originally raise in clair)

Add a 'Golang core' scanner to Clair. Because Golang binaries are statically compiled it can be hard to track and fix security vulnerabilities which are present in them. This feature aims to improve the situation by identifying Golang binaries in a layer, by using Golang compiler version the binary was compiled with, and matching that to NIST NVD data using the cpe:2.3:a:golang:go:*:*:*:*:*:*:*:* cpe.

A distribution scanner should be added which inspects every file in a docker layer using the linux tools, 'file', and 'gdb' to determine the Golang compiler version used to compile the binary.

file <target-file> | grep ELK
gdb -q -ex "p 'runtime.buildVersion'" -ex quit <target-file> 2>/dev/null

A new vulnsrc should be added which uses NIST NVD 1.1 Json data, filtered by Golang CPE. In that data, it's possible to identify which version addresses the vulnerability. For example CVE-2015-5741 affected golang up to, but excluding 1.4.3, from the 2015 data.

{
    "cve" : {
      "data_type" : "CVE",
      "data_format" : "MITRE",
      "data_version" : "4.0",
      "CVE_data_meta" : {
        "ID" : "CVE-2015-5741",
        "ASSIGNER" : "[email protected]"
      },
      "problemtype" : {
        "problemtype_data" : [ {
          "description" : [ {
            "lang" : "en",
            "value" : "CWE-444"
          } ]
        } ]
      },
... 
   "configurations" : {
      "CVE_data_version" : "4.0",
      "nodes" : [ {
        "operator" : "OR",
        "cpe_match" : [ {
          "vulnerable" : true,
          "cpe23Uri" : "cpe:2.3:a:golang:go:*:*:*:*:*:*:*:*",
          "versionEndExcluding" : "1.4.3"
        } ]
      }

java: flaky test

This is the second time I've seen this test flake like this:

2021-08-12T20:48:02.6671746Z go test -count=1 -race -tags integration ./...
2021-08-12T20:48:35.3849602Z ok  	github.com/quay/claircore	20.271s
2021-08-12T20:48:35.3852679Z ok  	github.com/quay/claircore/alpine	2.353s
2021-08-12T20:48:35.3855522Z ok  	github.com/quay/claircore/aws	13.908s
2021-08-12T20:48:47.9081914Z ?   	github.com/quay/claircore/cmd/cctool	[no test files]
2021-08-12T20:48:49.8043136Z ?   	github.com/quay/claircore/cmd/libindexhttp	[no test files]
2021-08-12T20:48:49.8043995Z ?   	github.com/quay/claircore/cmd/libvulnhttp	[no test files]
2021-08-12T20:51:49.2477084Z ok  	github.com/quay/claircore/debian	178.624s
2021-08-12T20:51:49.2478014Z ok  	github.com/quay/claircore/docs	0.024s [no tests to run]
2021-08-12T20:51:49.2478703Z ok  	github.com/quay/claircore/dpkg	35.377s
2021-08-12T20:51:49.7932806Z ok  	github.com/quay/claircore/enricher/cvss	142.206s
2021-08-12T20:51:49.7933625Z ?   	github.com/quay/claircore/internal/indexer	[no test files]
2021-08-12T20:51:50.2851308Z ok  	github.com/quay/claircore/internal/indexer/controller	0.046s
2021-08-12T20:51:50.9612048Z ok  	github.com/quay/claircore/internal/indexer/fetcher	0.155s
2021-08-12T20:51:51.3263314Z ok  	github.com/quay/claircore/internal/indexer/layerscanner	0.037s
2021-08-12T20:51:51.6377794Z ok  	github.com/quay/claircore/internal/indexer/linux	0.034s
2021-08-12T20:52:04.6834697Z ok  	github.com/quay/claircore/internal/indexer/postgres	11.901s
2021-08-12T20:52:04.6835576Z ?   	github.com/quay/claircore/internal/matcher	[no test files]
2021-08-12T20:52:04.6836387Z ?   	github.com/quay/claircore/internal/vulnstore	[no test files]
2021-08-12T20:52:04.6837244Z ok  	github.com/quay/claircore/internal/vulnstore/postgres	7.588s
2021-08-12T20:52:16.2660726Z --- FAIL: TestScan (14.21s)
2021-08-12T20:52:16.2662193Z     --- FAIL: TestScan/gremlin-console (14.21s)
2021-08-12T20:52:16.2664723Z         testing.go:88: {"level":"debug","component":"java/Scanner.Scan","layer":"sha256:e6e10dd7da4509f51dbdaf50f9d786f4ca0096ba14bfa7443f19d032e35f73f7","version":"2","zlog.testname":"TestScan/gremlin-console","caller":"packagescanner.go:55","message":"start"}
2021-08-12T20:52:16.2668942Z         testing.go:88: {"level":"debug","component":"java/Scanner.Scan","layer":"sha256:e6e10dd7da4509f51dbdaf50f9d786f4ca0096ba14bfa7443f19d032e35f73f7","version":"2","zlog.testname":"TestScan/gremlin-console","file":"opt/gremlin-console/ext/gremlin-groovy/lib/gremlin-groovy-3.4.8.jar","caller":"common.go:28","message":"found jar"}
2021-08-12T20:52:16.2673458Z         testing.go:88: {"level":"debug","component":"java/Scanner.Scan","layer":"sha256:e6e10dd7da4509f51dbdaf50f9d786f4ca0096ba14bfa7443f19d032e35f73f7","version":"2","zlog.testname":"TestScan/gremlin-console","file":"opt/gremlin-console/ext/gremlin-groovy/plugin/gremlin-groovy-3.4.8.jar","caller":"common.go:28","message":"found jar"}
2021-08-12T20:52:16.2678133Z         testing.go:88: {"level":"debug","component":"java/Scanner.Scan","layer":"sha256:e6e10dd7da4509f51dbdaf50f9d786f4ca0096ba14bfa7443f19d032e35f73f7","version":"2","zlog.testname":"TestScan/gremlin-console","file":"opt/gremlin-console/ext/tinkergraph-gremlin/lib/tinkergraph-gremlin-3.4.8.jar","caller":"common.go:28","message":"found jar"}
2021-08-12T20:52:16.2683220Z         testing.go:88: {"level":"debug","component":"java/Scanner.Scan","layer":"sha256:e6e10dd7da4509f51dbdaf50f9d786f4ca0096ba14bfa7443f19d032e35f73f7","version":"2","zlog.testname":"TestScan/gremlin-console","file":"opt/gremlin-console/ext/tinkergraph-gremlin/plugin/tinkergraph-gremlin-3.4.8.jar","caller":"common.go:28","message":"found jar"}
2021-08-12T20:52:16.2688602Z         testing.go:88: {"level":"debug","component":"java/Scanner.Scan","layer":"sha256:e6e10dd7da4509f51dbdaf50f9d786f4ca0096ba14bfa7443f19d032e35f73f7","version":"2","zlog.testname":"TestScan/gremlin-console","file":"opt/gremlin-console/lib/caffeine-2.3.1.jar","caller":"common.go:28","message":"found jar"}
2021-08-12T20:52:16.2692913Z         testing.go:88: {"level":"debug","component":"java/Scanner.Scan","layer":"sha256:e6e10dd7da4509f51dbdaf50f9d786f4ca0096ba14bfa7443f19d032e35f73f7","version":"2","zlog.testname":"TestScan/gremlin-console","file":"opt/gremlin-console/lib/commons-codec-1.14.jar","caller":"common.go:28","message":"found jar"}
2021-08-12T20:52:16.2697230Z         testing.go:88: {"level":"debug","component":"java/Scanner.Scan","layer":"sha256:e6e10dd7da4509f51dbdaf50f9d786f4ca0096ba14bfa7443f19d032e35f73f7","version":"2","zlog.testname":"TestScan/gremlin-console","file":"opt/gremlin-console/lib/commons-collections-3.2.2.jar","caller":"common.go:28","message":"found jar"}
2021-08-12T20:52:16.2703936Z         testing.go:88: {"level":"debug","component":"java/Scanner.Scan","layer":"sha256:e6e10dd7da4509f51dbdaf50f9d786f4ca0096ba14bfa7443f19d032e35f73f7","version":"2","zlog.testname":"TestScan/gremlin-console","file":"opt/gremlin-console/lib/commons-configuration-1.10.jar","caller":"common.go:28","message":"found jar"}
2021-08-12T20:52:16.2714146Z         testing.go:88: {"level":"debug","component":"java/Scanner.Scan","layer":"sha256:e6e10dd7da4509f51dbdaf50f9d786f4ca0096ba14bfa7443f19d032e35f73f7","version":"2","zlog.testname":"TestScan/gremlin-console","file":"opt/gremlin-console/lib/commons-lang-2.6.jar","caller":"common.go:28","message":"found jar"}
2021-08-12T20:52:16.2718242Z         testing.go:88: {"level":"debug","component":"java/Scanner.Scan","layer":"sha256:e6e10dd7da4509f51dbdaf50f9d786f4ca0096ba14bfa7443f19d032e35f73f7","version":"2","zlog.testname":"TestScan/gremlin-console","file":"opt/gremlin-console/lib/commons-lang3-3.8.1.jar","caller":"common.go:28","message":"found jar"}
2021-08-12T20:52:16.2722313Z         testing.go:88: {"level":"debug","component":"java/Scanner.Scan","layer":"sha256:e6e10dd7da4509f51dbdaf50f9d786f4ca0096ba14bfa7443f19d032e35f73f7","version":"2","zlog.testname":"TestScan/gremlin-console","file":"opt/gremlin-console/lib/commons-logging-1.2.jar","caller":"common.go:28","message":"found jar"}
2021-08-12T20:52:16.2726275Z         testing.go:88: {"level":"debug","component":"java/Scanner.Scan","layer":"sha256:e6e10dd7da4509f51dbdaf50f9d786f4ca0096ba14bfa7443f19d032e35f73f7","version":"2","zlog.testname":"TestScan/gremlin-console","file":"opt/gremlin-console/lib/exp4j-0.4.8.jar","caller":"common.go:28","message":"found jar"}
2021-08-12T20:52:16.2730336Z         testing.go:88: {"level":"debug","component":"java/Scanner.Scan","layer":"sha256:e6e10dd7da4509f51dbdaf50f9d786f4ca0096ba14bfa7443f19d032e35f73f7","version":"2","zlog.testname":"TestScan/gremlin-console","file":"opt/gremlin-console/lib/gremlin-console-3.4.8.jar","caller":"common.go:28","message":"found jar"}
2021-08-12T20:52:16.2734523Z         testing.go:88: {"level":"debug","component":"java/Scanner.Scan","layer":"sha256:e6e10dd7da4509f51dbdaf50f9d786f4ca0096ba14bfa7443f19d032e35f73f7","version":"2","zlog.testname":"TestScan/gremlin-console","file":"opt/gremlin-console/lib/gremlin-core-3.4.8.jar","caller":"common.go:28","message":"found jar"}
2021-08-12T20:52:16.2738614Z         testing.go:88: {"level":"debug","component":"java/Scanner.Scan","layer":"sha256:e6e10dd7da4509f51dbdaf50f9d786f4ca0096ba14bfa7443f19d032e35f73f7","version":"2","zlog.testname":"TestScan/gremlin-console","file":"opt/gremlin-console/lib/gremlin-driver-3.4.8.jar","caller":"common.go:28","message":"found jar"}
2021-08-12T20:52:16.2762322Z         testing.go:88: {"level":"debug","component":"java/Scanner.Scan","layer":"sha256:e6e10dd7da4509f51dbdaf50f9d786f4ca0096ba14bfa7443f19d032e35f73f7","version":"2","zlog.testname":"TestScan/gremlin-console","file":"opt/gremlin-console/lib/gremlin-shaded-3.4.8.jar","caller":"common.go:28","message":"found jar"}
2021-08-12T20:52:16.2766619Z         testing.go:88: {"level":"debug","component":"java/Scanner.Scan","layer":"sha256:e6e10dd7da4509f51dbdaf50f9d786f4ca0096ba14bfa7443f19d032e35f73f7","version":"2","zlog.testname":"TestScan/gremlin-console","file":"opt/gremlin-console/lib/groovy-2.5.11-indy.jar","caller":"common.go:28","message":"found jar"}
2021-08-12T20:52:16.2770761Z         testing.go:88: {"level":"debug","component":"java/Scanner.Scan","layer":"sha256:e6e10dd7da4509f51dbdaf50f9d786f4ca0096ba14bfa7443f19d032e35f73f7","version":"2","zlog.testname":"TestScan/gremlin-console","file":"opt/gremlin-console/lib/groovy-cli-picocli-2.5.11.jar","caller":"common.go:28","message":"found jar"}
2021-08-12T20:52:16.2774937Z         testing.go:88: {"level":"debug","component":"java/Scanner.Scan","layer":"sha256:e6e10dd7da4509f51dbdaf50f9d786f4ca0096ba14bfa7443f19d032e35f73f7","version":"2","zlog.testname":"TestScan/gremlin-console","file":"opt/gremlin-console/lib/groovy-console-2.5.11.jar","caller":"common.go:28","message":"found jar"}
2021-08-12T20:52:16.2779173Z         testing.go:88: {"level":"debug","component":"java/Scanner.Scan","layer":"sha256:e6e10dd7da4509f51dbdaf50f9d786f4ca0096ba14bfa7443f19d032e35f73f7","version":"2","zlog.testname":"TestScan/gremlin-console","file":"opt/gremlin-console/lib/groovy-groovysh-2.5.11-indy.jar","caller":"common.go:28","message":"found jar"}
2021-08-12T20:52:16.2784359Z         testing.go:88: {"level":"debug","component":"java/Scanner.Scan","layer":"sha256:e6e10dd7da4509f51dbdaf50f9d786f4ca0096ba14bfa7443f19d032e35f73f7","version":"2","zlog.testname":"TestScan/gremlin-console","file":"opt/gremlin-console/lib/groovy-json-2.5.11-indy.jar","caller":"common.go:28","message":"found jar"}
2021-08-12T20:52:16.2788490Z         testing.go:88: {"level":"debug","component":"java/Scanner.Scan","layer":"sha256:e6e10dd7da4509f51dbdaf50f9d786f4ca0096ba14bfa7443f19d032e35f73f7","version":"2","zlog.testname":"TestScan/gremlin-console","file":"opt/gremlin-console/lib/groovy-jsr223-2.5.11-indy.jar","caller":"common.go:28","message":"found jar"}
2021-08-12T20:52:16.2792585Z         testing.go:88: {"level":"debug","component":"java/Scanner.Scan","layer":"sha256:e6e10dd7da4509f51dbdaf50f9d786f4ca0096ba14bfa7443f19d032e35f73f7","version":"2","zlog.testname":"TestScan/gremlin-console","file":"opt/gremlin-console/lib/groovy-swing-2.5.11.jar","caller":"common.go:28","message":"found jar"}
2021-08-12T20:52:16.2796687Z         testing.go:88: {"level":"debug","component":"java/Scanner.Scan","layer":"sha256:e6e10dd7da4509f51dbdaf50f9d786f4ca0096ba14bfa7443f19d032e35f73f7","version":"2","zlog.testname":"TestScan/gremlin-console","file":"opt/gremlin-console/lib/groovy-templates-2.5.11.jar","caller":"common.go:28","message":"found jar"}
2021-08-12T20:52:16.2800736Z         testing.go:88: {"level":"debug","component":"java/Scanner.Scan","layer":"sha256:e6e10dd7da4509f51dbdaf50f9d786f4ca0096ba14bfa7443f19d032e35f73f7","version":"2","zlog.testname":"TestScan/gremlin-console","file":"opt/gremlin-console/lib/groovy-xml-2.5.11.jar","caller":"common.go:28","message":"found jar"}
2021-08-12T20:52:16.2804635Z         testing.go:88: {"level":"debug","component":"java/Scanner.Scan","layer":"sha256:e6e10dd7da4509f51dbdaf50f9d786f4ca0096ba14bfa7443f19d032e35f73f7","version":"2","zlog.testname":"TestScan/gremlin-console","file":"opt/gremlin-console/lib/hppc-0.7.1.jar","caller":"common.go:28","message":"found jar"}
2021-08-12T20:52:16.2808536Z         testing.go:88: {"level":"debug","component":"java/Scanner.Scan","layer":"sha256:e6e10dd7da4509f51dbdaf50f9d786f4ca0096ba14bfa7443f19d032e35f73f7","version":"2","zlog.testname":"TestScan/gremlin-console","file":"opt/gremlin-console/lib/httpclient-4.5.8.jar","caller":"common.go:28","message":"found jar"}
2021-08-12T20:52:16.2812607Z         testing.go:88: {"level":"debug","component":"java/Scanner.Scan","layer":"sha256:e6e10dd7da4509f51dbdaf50f9d786f4ca0096ba14bfa7443f19d032e35f73f7","version":"2","zlog.testname":"TestScan/gremlin-console","file":"opt/gremlin-console/lib/httpcore-4.4.11.jar","caller":"common.go:28","message":"found jar"}
2021-08-12T20:52:16.2816557Z         testing.go:88: {"level":"debug","component":"java/Scanner.Scan","layer":"sha256:e6e10dd7da4509f51dbdaf50f9d786f4ca0096ba14bfa7443f19d032e35f73f7","version":"2","zlog.testname":"TestScan/gremlin-console","file":"opt/gremlin-console/lib/ivy-2.3.0.jar","caller":"common.go:28","message":"found jar"}
2021-08-12T20:52:16.2841300Z         testing.go:88: {"level":"debug","component":"java/Scanner.Scan","layer":"sha256:e6e10dd7da4509f51dbdaf50f9d786f4ca0096ba14bfa7443f19d032e35f73f7","version":"2","zlog.testname":"TestScan/gremlin-console","file":"opt/gremlin-console/lib/javapoet-1.8.0.jar","caller":"common.go:28","message":"found jar"}
2021-08-12T20:52:16.2845809Z         testing.go:88: {"level":"debug","component":"java/Scanner.Scan","layer":"sha256:e6e10dd7da4509f51dbdaf50f9d786f4ca0096ba14bfa7443f19d032e35f73f7","version":"2","zlog.testname":"TestScan/gremlin-console","file":"opt/gremlin-console/lib/javatuples-1.2.jar","caller":"common.go:28","message":"found jar"}
2021-08-12T20:52:16.2849741Z         testing.go:88: {"level":"debug","component":"java/Scanner.Scan","layer":"sha256:e6e10dd7da4509f51dbdaf50f9d786f4ca0096ba14bfa7443f19d032e35f73f7","version":"2","zlog.testname":"TestScan/gremlin-console","file":"opt/gremlin-console/lib/jbcrypt-0.4.jar","caller":"common.go:28","message":"found jar"}
2021-08-12T20:52:16.2853646Z         testing.go:88: {"level":"debug","component":"java/Scanner.Scan","layer":"sha256:e6e10dd7da4509f51dbdaf50f9d786f4ca0096ba14bfa7443f19d032e35f73f7","version":"2","zlog.testname":"TestScan/gremlin-console","file":"opt/gremlin-console/lib/jcabi-log-0.14.jar","caller":"common.go:28","message":"found jar"}
2021-08-12T20:52:16.2857665Z         testing.go:88: {"level":"debug","component":"java/Scanner.Scan","layer":"sha256:e6e10dd7da4509f51dbdaf50f9d786f4ca0096ba14bfa7443f19d032e35f73f7","version":"2","zlog.testname":"TestScan/gremlin-console","file":"opt/gremlin-console/lib/jcabi-manifests-1.1.jar","caller":"common.go:28","message":"found jar"}
2021-08-12T20:52:16.2861920Z         testing.go:88: {"level":"debug","component":"java/Scanner.Scan","layer":"sha256:e6e10dd7da4509f51dbdaf50f9d786f4ca0096ba14bfa7443f19d032e35f73f7","version":"2","zlog.testname":"TestScan/gremlin-console","file":"opt/gremlin-console/lib/jcl-over-slf4j-1.7.25.jar","caller":"common.go:28","message":"found jar"}
2021-08-12T20:52:16.2865912Z         testing.go:88: {"level":"debug","component":"java/Scanner.Scan","layer":"sha256:e6e10dd7da4509f51dbdaf50f9d786f4ca0096ba14bfa7443f19d032e35f73f7","version":"2","zlog.testname":"TestScan/gremlin-console","file":"opt/gremlin-console/lib/jline-2.14.6.jar","caller":"common.go:28","message":"found jar"}
2021-08-12T20:52:16.2869792Z         testing.go:88: {"level":"debug","component":"java/Scanner.Scan","layer":"sha256:e6e10dd7da4509f51dbdaf50f9d786f4ca0096ba14bfa7443f19d032e35f73f7","version":"2","zlog.testname":"TestScan/gremlin-console","file":"opt/gremlin-console/lib/log4j-1.2.17.jar","caller":"common.go:28","message":"found jar"}
2021-08-12T20:52:16.2873749Z         testing.go:88: {"level":"debug","component":"java/Scanner.Scan","layer":"sha256:e6e10dd7da4509f51dbdaf50f9d786f4ca0096ba14bfa7443f19d032e35f73f7","version":"2","zlog.testname":"TestScan/gremlin-console","file":"opt/gremlin-console/lib/netty-all-4.1.49.Final.jar","caller":"common.go:28","message":"found jar"}
2021-08-12T20:52:16.2877759Z         testing.go:88: {"level":"debug","component":"java/Scanner.Scan","layer":"sha256:e6e10dd7da4509f51dbdaf50f9d786f4ca0096ba14bfa7443f19d032e35f73f7","version":"2","zlog.testname":"TestScan/gremlin-console","file":"opt/gremlin-console/lib/picocli-4.0.1.jar","caller":"common.go:28","message":"found jar"}
2021-08-12T20:52:16.2881914Z         testing.go:88: {"level":"debug","component":"java/Scanner.Scan","layer":"sha256:e6e10dd7da4509f51dbdaf50f9d786f4ca0096ba14bfa7443f19d032e35f73f7","version":"2","zlog.testname":"TestScan/gremlin-console","file":"opt/gremlin-console/lib/slf4j-api-1.7.25.jar","caller":"common.go:28","message":"found jar"}
2021-08-12T20:52:16.2885975Z         testing.go:88: {"level":"debug","component":"java/Scanner.Scan","layer":"sha256:e6e10dd7da4509f51dbdaf50f9d786f4ca0096ba14bfa7443f19d032e35f73f7","version":"2","zlog.testname":"TestScan/gremlin-console","file":"opt/gremlin-console/lib/slf4j-log4j12-1.7.25.jar","caller":"common.go:28","message":"found jar"}
2021-08-12T20:52:16.2889930Z         testing.go:88: {"level":"debug","component":"java/Scanner.Scan","layer":"sha256:e6e10dd7da4509f51dbdaf50f9d786f4ca0096ba14bfa7443f19d032e35f73f7","version":"2","zlog.testname":"TestScan/gremlin-console","file":"opt/gremlin-console/lib/snakeyaml-1.15.jar","caller":"common.go:28","message":"found jar"}
2021-08-12T20:52:16.2893644Z         testing.go:88: {"level":"debug","component":"java/Scanner.Scan","layer":"sha256:e6e10dd7da4509f51dbdaf50f9d786f4ca0096ba14bfa7443f19d032e35f73f7","version":"2","zlog.testname":"TestScan/gremlin-console","caller":"packagescanner.go:93","message":"done"}
2021-08-12T20:52:16.2895561Z         packagescanner.go:59: found 36 packages
2021-08-12T20:52:16.2897641Z         packagescanner.go:61:   []*claircore.Package{
2021-08-12T20:52:16.2898607Z               	... // 2 identical elements
2021-08-12T20:52:16.2900129Z               	&{Name: "org.apache.tinkerpop:tinkergraph-gremlin", Version: "3.4.8", Kind: "binary", PackageDB: "maven:opt/gremlin-console/ext/tinkergraph-gremlin/lib", ...},
2021-08-12T20:52:16.2903195Z               	&{Name: "org.apache.tinkerpop:tinkergraph-gremlin", Version: "3.4.8", Kind: "binary", PackageDB: "maven:opt/gremlin-console/ext/tinkergraph-gremlin/plugin", ...},
2021-08-12T20:52:16.2904532Z             - 	&{
2021-08-12T20:52:16.2905422Z             - 		Name:           "com.github.ben-manes.caffeine:caffeine",
2021-08-12T20:52:16.2906332Z             - 		Version:        "2.3.1",
2021-08-12T20:52:16.2906953Z             - 		Kind:           "binary",
2021-08-12T20:52:16.2907748Z             - 		PackageDB:      "maven:opt/gremlin-console/lib",
2021-08-12T20:52:16.2908964Z             - 		RepositoryHint: "https://repo1.maven.apache.org/maven2",
2021-08-12T20:52:16.2909863Z             - 	},
2021-08-12T20:52:16.2910973Z               	&{Name: "commons-codec:commons-codec", Version: "1.14", Kind: "binary", PackageDB: "maven:opt/gremlin-console/lib", ...},
2021-08-12T20:52:16.2912651Z               	&{Name: "commons-collections:commons-collections", Version: "3.2.2", Kind: "binary", PackageDB: "maven:opt/gremlin-console/lib", ...},
2021-08-12T20:52:16.2913854Z               	... // 13 identical elements
2021-08-12T20:52:16.2915249Z               	&{Name: "com.fasterxml.jackson.core:jackson-annotations", Version: "2.9.10", Kind: "binary", PackageDB: "maven:opt/gremlin-console/lib", ...},
2021-08-12T20:52:16.2917106Z               	&{Name: "com.fasterxml.jackson.core:jackson-core", Version: "2.9.10", Kind: "binary", PackageDB: "maven:opt/gremlin-console/lib", ...},
2021-08-12T20:52:16.2918184Z             - 	&{
2021-08-12T20:52:16.2918901Z             - 		Name:           "org.codehaus.groovy:groovy",
2021-08-12T20:52:16.2919668Z             - 		Version:        "2.5.11",
2021-08-12T20:52:16.2920267Z             - 		Kind:           "binary",
2021-08-12T20:52:16.2921063Z             - 		PackageDB:      "maven:opt/gremlin-console/lib",
2021-08-12T20:52:16.2922234Z             - 		RepositoryHint: "https://repo1.maven.apache.org/maven2",
2021-08-12T20:52:16.2923140Z             - 	},
2021-08-12T20:52:16.2923650Z             - 	&{
2021-08-12T20:52:16.2924662Z             - 		Name:           "org.codehaus.groovy:groovy-cli-picocli",
2021-08-12T20:52:16.2925518Z             - 		Version:        "2.5.11",
2021-08-12T20:52:16.2926120Z             - 		Kind:           "binary",
2021-08-12T20:52:16.2926926Z             - 		PackageDB:      "maven:opt/gremlin-console/lib",
2021-08-12T20:52:16.2928095Z             - 		RepositoryHint: "https://repo1.maven.apache.org/maven2",
2021-08-12T20:52:16.2928997Z             - 	},
2021-08-12T20:52:16.2929472Z             - 	&{
2021-08-12T20:52:16.2930360Z             - 		Name:           "org.codehaus.groovy:groovy-console",
2021-08-12T20:52:16.2931186Z             - 		Version:        "2.5.11",
2021-08-12T20:52:16.2931789Z             - 		Kind:           "binary",
2021-08-12T20:52:16.2932588Z             - 		PackageDB:      "maven:opt/gremlin-console/lib",
2021-08-12T20:52:16.2933758Z             - 		RepositoryHint: "https://repo1.maven.apache.org/maven2",
2021-08-12T20:52:16.2934669Z             - 	},
2021-08-12T20:52:16.2935154Z             - 	&{
2021-08-12T20:52:16.2935952Z             - 		Name:           "org.codehaus.groovy:groovy-groovysh",
2021-08-12T20:52:16.2936760Z             - 		Version:        "2.5.11",
2021-08-12T20:52:16.2937371Z             - 		Kind:           "binary",
2021-08-12T20:52:16.2938156Z             - 		PackageDB:      "maven:opt/gremlin-console/lib",
2021-08-12T20:52:16.2939333Z             - 		RepositoryHint: "https://repo1.maven.apache.org/maven2",
2021-08-12T20:52:16.2940240Z             - 	},
2021-08-12T20:52:16.2940903Z             - 	&{
2021-08-12T20:52:16.2941681Z             - 		Name:           "org.codehaus.groovy:groovy-json",
2021-08-12T20:52:16.2942456Z             - 		Version:        "2.5.11",
2021-08-12T20:52:16.2943072Z             - 		Kind:           "binary",
2021-08-12T20:52:16.2943857Z             - 		PackageDB:      "maven:opt/gremlin-console/lib",
2021-08-12T20:52:16.2945031Z             - 		RepositoryHint: "https://repo1.maven.apache.org/maven2",
2021-08-12T20:52:16.2945916Z             - 	},
2021-08-12T20:52:16.2946410Z             - 	&{
2021-08-12T20:52:16.2947174Z             - 		Name:           "org.codehaus.groovy:groovy-jsr223",
2021-08-12T20:52:16.2947978Z             - 		Version:        "2.5.11",
2021-08-12T20:52:16.2948592Z             - 		Kind:           "binary",
2021-08-12T20:52:16.2949373Z             - 		PackageDB:      "maven:opt/gremlin-console/lib",
2021-08-12T20:52:16.2950557Z             - 		RepositoryHint: "https://repo1.maven.apache.org/maven2",
2021-08-12T20:52:16.2951459Z             - 	},
2021-08-12T20:52:16.2951942Z             - 	&{
2021-08-12T20:52:16.2952707Z             - 		Name:           "org.codehaus.groovy:groovy-swing",
2021-08-12T20:52:16.2953501Z             - 		Version:        "2.5.11",
2021-08-12T20:52:16.2954098Z             - 		Kind:           "binary",
2021-08-12T20:52:16.2954894Z             - 		PackageDB:      "maven:opt/gremlin-console/lib",
2021-08-12T20:52:16.2956057Z             - 		RepositoryHint: "https://repo1.maven.apache.org/maven2",
2021-08-12T20:52:16.2956974Z             - 	},
2021-08-12T20:52:16.2957461Z             - 	&{
2021-08-12T20:52:16.2958267Z             - 		Name:           "org.codehaus.groovy:groovy-templates",
2021-08-12T20:52:16.2959104Z             - 		Version:        "2.5.11",
2021-08-12T20:52:16.2959700Z             - 		Kind:           "binary",
2021-08-12T20:52:16.2960494Z             - 		PackageDB:      "maven:opt/gremlin-console/lib",
2021-08-12T20:52:16.2961662Z             - 		RepositoryHint: "https://repo1.maven.apache.org/maven2",
2021-08-12T20:52:16.2962560Z             - 	},
2021-08-12T20:52:16.2963039Z             - 	&{
2021-08-12T20:52:16.2963788Z             - 		Name:           "org.codehaus.groovy:groovy-xml",
2021-08-12T20:52:16.2964562Z             - 		Version:        "2.5.11",
2021-08-12T20:52:16.2965160Z             - 		Kind:           "binary",
2021-08-12T20:52:16.2965952Z             - 		PackageDB:      "maven:opt/gremlin-console/lib",
2021-08-12T20:52:16.2967143Z             - 		RepositoryHint: "https://repo1.maven.apache.org/maven2",
2021-08-12T20:52:16.2968188Z             - 	},
2021-08-12T20:52:16.2969228Z               	&{Name: "com.carrotsearch:hppc", Version: "0.7.1", Kind: "binary", PackageDB: "maven:opt/gremlin-console/lib", ...},
2021-08-12T20:52:16.2970860Z               	&{Name: "org.apache.httpcomponents:httpclient", Version: "4.5.8", Kind: "binary", PackageDB: "maven:opt/gremlin-console/lib", ...},
2021-08-12T20:52:16.2972619Z               	&{Name: "org.apache.httpcomponents:httpcore", Version: "4.4.11", Kind: "binary", PackageDB: "maven:opt/gremlin-console/lib", ...},
2021-08-12T20:52:16.2973888Z             - 	&{
2021-08-12T20:52:16.2974548Z             - 		Name:           "org.apache.ivy:ivy",
2021-08-12T20:52:16.2975237Z             - 		Version:        "2.3.0",
2021-08-12T20:52:16.2975833Z             - 		Kind:           "binary",
2021-08-12T20:52:16.2976670Z             - 		PackageDB:      "maven:opt/gremlin-console/lib",
2021-08-12T20:52:16.2977848Z             - 		RepositoryHint: "https://repo1.maven.apache.org/maven2",
2021-08-12T20:52:16.2978773Z             - 	},
2021-08-12T20:52:16.2979795Z               	&{Name: "com.squareup:javapoet", Version: "1.8.0", Kind: "binary", PackageDB: "maven:opt/gremlin-console/lib", ...},
2021-08-12T20:52:16.2981756Z               	&{Name: "org.javatuples:javatuples", Version: "1.2", Kind: "binary", PackageDB: "maven:opt/gremlin-console/lib", ...},
2021-08-12T20:52:16.2982814Z               	... // 5 identical elements
2021-08-12T20:52:16.2983865Z               	&{Name: "log4j:log4j", Version: "1.2.17", Kind: "binary", PackageDB: "maven:opt/gremlin-console/lib", ...},
2021-08-12T20:52:16.2985237Z               	&{Name: "io.netty:netty-all", Version: "4.1.49.Final", Kind: "binary", PackageDB: "maven:opt/gremlin-console/lib", ...},
2021-08-12T20:52:16.2986116Z             - 	&{
2021-08-12T20:52:16.2986781Z             - 		Name:           "info.picocli:picocli",
2021-08-12T20:52:16.2987471Z             - 		Version:        "4.0.1",
2021-08-12T20:52:16.2988083Z             - 		Kind:           "binary",
2021-08-12T20:52:16.2988881Z             - 		PackageDB:      "maven:opt/gremlin-console/lib",
2021-08-12T20:52:16.2990049Z             - 		RepositoryHint: "https://repo1.maven.apache.org/maven2",
2021-08-12T20:52:16.2990954Z             - 	},
2021-08-12T20:52:16.2991948Z               	&{Name: "org.slf4j:slf4j-api", Version: "1.7.25", Kind: "binary", PackageDB: "maven:opt/gremlin-console/lib", ...},
2021-08-12T20:52:16.2993370Z               	&{Name: "org.slf4j:slf4j-log4j12", Version: "1.7.25", Kind: "binary", PackageDB: "maven:opt/gremlin-console/lib", ...},
2021-08-12T20:52:16.2994766Z               	&{Name: "org.yaml:snakeyaml", Version: "1.15", Kind: "binary", PackageDB: "maven:opt/gremlin-console/lib", ...},
2021-08-12T20:52:16.2995609Z               }
2021-08-12T20:52:16.2995956Z             
2021-08-12T20:52:16.2996289Z FAIL
2021-08-12T20:52:16.2996781Z FAIL	github.com/quay/claircore/java	14.249s
2021-08-12T20:52:16.2997413Z ok  	github.com/quay/claircore/libindex	5.606s
2021-08-12T20:52:16.2998174Z ?   	github.com/quay/claircore/libindex/migrations	[no test files]
2021-08-12T20:52:16.2998926Z ?   	github.com/quay/claircore/libvuln	[no test files]
2021-08-12T20:52:16.2999663Z ?   	github.com/quay/claircore/libvuln/driver	[no test files]
2021-08-12T20:52:16.3000388Z ok  	github.com/quay/claircore/libvuln/jsonblob	0.065s
2021-08-12T20:52:16.3001144Z ?   	github.com/quay/claircore/libvuln/migrations	[no test files]
2021-08-12T20:52:16.3002024Z ok  	github.com/quay/claircore/libvuln/updates	0.037s
2021-08-12T20:52:16.3002738Z ?   	github.com/quay/claircore/matchers	[no test files]
2021-08-12T20:52:16.3003503Z ?   	github.com/quay/claircore/matchers/defaults	[no test files]
2021-08-12T20:52:16.3004292Z ?   	github.com/quay/claircore/matchers/registry	[no test files]
2021-08-12T20:52:18.7403707Z ok  	github.com/quay/claircore/oracle	4.560s
2021-08-12T20:52:35.3776520Z ok  	github.com/quay/claircore/osrelease	18.208s
2021-08-12T20:53:57.1339444Z ok  	github.com/quay/claircore/photon	97.504s
2021-08-12T20:53:57.1340659Z ok  	github.com/quay/claircore/pkg/cpe	0.042s
2021-08-12T20:53:57.1341368Z ?   	github.com/quay/claircore/pkg/distlock	[no test files]
2021-08-12T20:53:57.1342159Z ?   	github.com/quay/claircore/pkg/distlock/postgres	[no test files]
2021-08-12T20:53:57.1343387Z ok  	github.com/quay/claircore/pkg/fastesturl	0.039s
2021-08-12T20:53:57.1344104Z ?   	github.com/quay/claircore/pkg/jsonerr	[no test files]
2021-08-12T20:53:57.1344865Z ?   	github.com/quay/claircore/pkg/microbatch	[no test files]
2021-08-12T20:53:57.1345832Z ?   	github.com/quay/claircore/pkg/omnimatcher	[no test files]
2021-08-12T20:53:57.1346558Z ok  	github.com/quay/claircore/pkg/ovalutil	9.953s
2021-08-12T20:53:57.1347252Z ?   	github.com/quay/claircore/pkg/path	[no test files]
2021-08-12T20:53:57.1347931Z ok  	github.com/quay/claircore/pkg/pep440	0.050s
2021-08-12T20:53:57.1348613Z ?   	github.com/quay/claircore/pkg/tmp	[no test files]
2021-08-12T20:53:57.1349263Z ok  	github.com/quay/claircore/python	38.727s
2021-08-12T20:53:57.1349908Z ok  	github.com/quay/claircore/pyupio	0.055s
2021-08-12T20:53:57.1350523Z ok  	github.com/quay/claircore/rhel	20.496s
2021-08-12T20:53:57.1351323Z ?   	github.com/quay/claircore/rhel/containerapi	[no test files]
2021-08-12T20:53:57.1352323Z ?   	github.com/quay/claircore/rhel/contentmanifest	[no test files]
2021-08-12T20:53:57.1353139Z ok  	github.com/quay/claircore/rhel/pulp	0.041s
2021-08-12T20:53:57.1353854Z ?   	github.com/quay/claircore/rhel/repo2cpe	[no test files]
2021-08-12T20:54:44.2629655Z ok  	github.com/quay/claircore/rpm	52.412s
2021-08-12T20:54:44.2630483Z ok  	github.com/quay/claircore/scanner/pkgconfig	0.032s
2021-08-12T20:55:00.4160977Z ok  	github.com/quay/claircore/suse	61.691s
2021-08-12T20:55:00.4161772Z ?   	github.com/quay/claircore/test	[no test files]
2021-08-12T20:55:00.4163591Z ?   	github.com/quay/claircore/test/bisect	[no test files]
2021-08-12T20:55:00.4164339Z ?   	github.com/quay/claircore/test/fetch	[no test files]
2021-08-12T20:55:00.4165119Z ?   	github.com/quay/claircore/test/integration	[no test files]
2021-08-12T20:55:00.4165912Z ?   	github.com/quay/claircore/test/ovaldebug	[no test files]
2021-08-12T20:55:00.4166674Z ?   	github.com/quay/claircore/test/postgres	[no test files]
2021-08-12T20:55:00.4167372Z ok  	github.com/quay/claircore/ubuntu	0.072s
2021-08-12T20:55:00.4168039Z ?   	github.com/quay/claircore/updater	[no test files]
2021-08-12T20:55:00.4168876Z ?   	github.com/quay/claircore/updater/defaults	[no test files]
2021-08-12T20:55:00.4169446Z FAIL

Prow integration

per @jzelinskie we should look into integrating Prow into the repository.
https://github.com/kubernetes/test-infra/tree/master/prow

Curb until OLM team is reachable.

fixed_in_version value in Vulnerability database table containing space character causing "upstream_version includes invalid character" error for ubuntu image

The error is triggered here by the go-deb-version library which doesn't expect the version to include a space character. That library is called via the debian matcher

I've included a search in the vuln table in the datablse to show occurances with fixed_in_version containing a space character.

If you look at the ubuntu link https://ubuntu.com/security/CVE-2020-14760 for the CVE in that image you can see a reference to 5.7 only for the upstream version.

Now the question. Is the version validator wrong or should the updater parse this data differently?

Environment

Clair version/image: v4.1.0

Thank you

smarter disk allocation

In our current implementation, we assume that disk space is not a problem.

In highly-tuned or tightly packed containerized deployments, using the container's ephemeral disk may result in evictions.

We should have some limit to disk usage that is reasonable by default, tunes down automatically, and can be manually increased.

A call to syscall.Statfs could help determine sizing, and then something like a golang.org/x/sync/semaphore.Weighted could be used to control requests.

There's probably a heuristic around detecting a volume dedicated for our use and just having the ephemeral root.

Can't detect Alpine release version 3.11 and higher

Description of Problem / Feature Request

Clair v4.0.0-rc.19 doesn't detect vulnerabilities in Alpine images v3.11 and higher because alpine distribution scanner can't detect release version 3.11 and higher:

claircore/alpine/distributionscanner.go

Line 35 in 04cb53c

var alpineRegexes = []alpineRegex{

Expected Outcome

{"vulnerabilities":{"2467":{"id":"2467","updater":"alpine-main-v3.12-updater","name":"CVE-2016-4074","description":"","issued":"0001-01-01T00:00:00Z","links":"https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2016-4074","severity":"","normalized_severity":"Unknown","package":{"id":"","name":"jq","version":"","kind":"binary","normalized_version":"","cpe":""},"distribution":{"id":"","did":"alpine","name":"Alpine Linux","version":"","version_code_name":"","version_id":"3.12","arch":"","cpe":"","pretty_name":"Alpine Linux v3.12"},"repository":{"cpe":""},"fixed_in_version":"1.6_rc1-r0"}},"package_vulnerabilities":{"192":["2467"]}}

Actual Outcome

{"vulnerabilities":{},"package_vulnerabilities":{}}

Environment

Clair version/image: v4.0.0-rc.19
Clair client name/version: clairctl version 1.2.8
Host OS: "Ubuntu 20.04.1 LTS"
Kernel (e.g. uname -a): Linux 5.4.0-52-generic
Kubernetes version (use kubectl version): NA
Network/Firewall setup: NA

Alpine Matcher Incorrectly Matches Fixed_in_version

Description of Problem / Feature Request

Alpine matcher incorrectly checks fixed_in_version = "" and filters out vulnerabilities where fixed_in_version = 0.

To test this, I will scan a layered image with a vulnerable curl in CVE-2021-22897 (https://security.alpinelinux.org/vuln/CVE-2021-22897)

Dockerfile

FROM alpine:3.11
RUN apk update && \
    apk upgrade && \
    apk add curl=7.67.0-r5

Ensuring the vulnerability exists in clair-db for Alpine 3.11

clair=# select name, dist_id, dist_version_id, package_name, fixed_in_version from vuln where dist_id='alpine' and dist_version_id='3.11' and name='CVE-2021-22897';
      name      | dist_id | dist_version_id | package_name | fixed_in_version 
----------------+---------+-----------------+--------------+------------------
 CVE-2021-22897 | alpine  | 3.11            | curl         | 0
(1 row)

Expected Outcome

Clair VulnerabilityReport to contain curl as a package as well as report CVE-2201-22897 as vulnerability

Actual Outcome

Clair does find curl as a package

              '106': {'arch': 'x86_64',
                      'id': '106',
                      'kind': 'binary',
                      'name': 'libcurl',
                      'source': {'id': '105',
                                 'kind': 'source',
                                 'name': 'curl',
                                 'version': '7.67.0-r5'},
                      'version': '7.67.0-r5'},
              '108': {'arch': 'x86_64',
                      'id': '108',
                      'kind': 'binary',
                      'name': 'curl',
                      'source': {'id': '105',
                                 'kind': 'source',
                                 'name': 'curl',
                                 'version': '7.67.0-r5'},

However it does not return any vulnerabilities.
Note: No vulnerabilities were reported at all, not even the ones reported by the base image. This is because apk upgrade updated apk-tools, libcrypto, libssl to fixed versions.

Investigation

Digging through the code revealed that Claircore is filtering out vulnerabilities incorrectly in the alpine matcher (https://github.com/quay/claircore/blob/main/alpine/matcher.go#L53).

func (*Matcher) Vulnerable(ctx context.Context, record *claircore.IndexRecord, vuln *claircore.Vulnerability) (bool, error) {
    v1, err := version.NewVersion(record.Package.Version)
    if err != nil {
        return false, nil
    }

    v2, err := version.NewVersion(vuln.FixedInVersion)
    if err != nil {
        return false, nil
    }

    if vuln.FixedInVersion == "" {
        return true, nil
    }

    if v1.LessThan(v2) {
        return true, nil
    }

    return false, nil
}

Looks like it’s checking if fixed_in_version = "" instead of fixed_in_version="0"
Querying the database verified that there are no cases where fixed_in_version = "" for alpine vulnerabilities.

clair=# select name, dist_id, dist_version_id, package_name, fixed_in_version from vuln where dist_id='alpine' and fixed_in_version='';
 name | dist_id | dist_version_id | package_name | fixed_in_version 
------+---------+-----------------+--------------+------------------
(0 rows)

Environment

Clair version/image: Clair 4.2.0, Claircore 0.5.2
Clair client name/version: localhost
Host OS: AmazonLinux2
Kernel (e.g. uname -a):
Kubernetes version (use kubectl version): N/A
Network/Firewall setup: N/A

Ubuntu Precise and Artful urls 404

7:47PM WRN updater error error="updater ubuntu-precise-updater failed to update: failed to parse the fetched vulnerability database: failed to decode OVAL xml contents: expected element type <oval_definitions> but have <html>" component=libvuln/New
7:47PM WRN updater error error="updater ubuntu-artful-updater failed to update: failed to parse the fetched vulnerability database: failed to decode OVAL xml contents: expected element type <oval_definitions> but have <html>"

@alexmurray did ubuntu stop hosting these oval sec dbs? The root site no longer has directory listing and ClairCore is failing to retrieve these.

Add a Go modules scanner, and updater

For Go binaries built with go modules, GA in Go 1.14, it's possible to extract the module information from the binary using 'goversion -m'.

I'm not sure there is a currently a reliable source for vulnerabilities Golang libraries for an updater, perhaps from Github, like NextB's vulnerability scanner is doing?

IndexReport State Should be an Enum

Description

Clients will rely on the IndexReport.State field, which is currently typed as string. The documentation claims that the Indexer follows a finite state machine with defined states. We can use these as a typed Go string enum.

Photon-OS ClairCore Support

Hi, I have already spent a lot of time working on this pr for Clair: https://github.com/quay/clair/pull/886/files.

After contacting @ldelossa I was advised to move the pr here.

Can you please help me figure out how much work will that be?

Uses Red Hat's Oval v2 instead of v1

Red Hat recently published v2 stream of Oval data. It covers more than just RHEL product. The format should be the same as v1 with one exception - CPE list contains more accurate data. Clair core should switch to this data source to cover whole Red Hat's portfolio.

https://www.redhat.com/security/data/oval/v2/

https://www.redhat.com/security/data/oval/v2/PULP_MANIFEST should help with identifying which oval files are relevant.

Does there exist a "vendor-independent" format to report container content

Hi,

we are building in kubevirt containers without a base image but which are built out of RPMs (they are built in a reproducible way without dnf and rpm). Therefore we don't have a rpmdb to share for clair and we also don't ship the rpm tool inside the container.

Is there something like an independent json/yaml file format which we could use, so that clair could simploy copy that info out and interpret it?

Structured Logging

RPM command error when getting rpm packages from layer database

When scanning an image with rpm database I am getting the following error.

{"level":"warn","component":"init/NewHttpTransport","component":"libindex/Libindex.Index","manifest":"sha256:fc84b5febd328eccaa913807716887b3eb5ed08bc22cc6933a9ebf82766725e3","component":"libindex/Libindex.index","component":"internal/indexer/controller/Controller.Index","manifest":"sha256:fc84b5febd328eccaa913807716887b3eb5ed08bc22cc6933a9ebf82766725e3","state":"CheckManifest","component":"internal/indexer/layerscannner/layerScanner.Scan","manifest":"sha256:fc84b5febd328eccaa913807716887b3eb5ed08bc22cc6933a9ebf82766725e3","component":"internal/indexer/layerscannner/layerScanner.scan","scanner":"rpm","kind":"package","layer":"sha256:787f47dbeaac926cc8c927a48e8a84fc97e05fbd1c08a4c9c4ddcaddfd87e2f7","component":"rpm/Scanner.Scan","version":"v0.0.1","layer":"sha256:787f47dbeaac926cc8c927a48e8a84fc97e05fbd1c08a4c9c4ddcaddfd87e2f7","db":"/var/lib/rpm","cmd":["rpm","--root","/tmp/rpmscanner.012828750","--dbpath","/var/lib/rpm","--query","--all","--queryformat","%{name}\\n%{evr}\\n%{payloaddigestalgo}:%{payloaddigest}\\n%{sigpgp:pgpsig}\\n%{sourcerpm}\\n%{RPMTAG_MODULARITYLABEL}\\n%{ARCH}\\n.\\n"],"err":"error: db5 error(5) from dbenv->open: I/O error\nerror: cannot open Packages index using db5 - I/O error (5)\nerror: cannot open Packages database in /tmp/rpmscanner.012828750/var/lib/rpm\nerror: db5 error(5) from dbenv->open: I/O error\nerror: cannot open Packages index using db5 - I/O error (5)\nerror: cannot open Packages database in /tmp/rpmscanner.012828750/var/lib/rpm\n","time":"2020-07-03T12:15:51Z","time":"2020-07-03T12:15:51Z","message":"error output"}

Manifest reproducer:

{
  "hash": "sha256:fc84b5febd328eccaa913807716887b3eb5ed08bc22cc6933a9ebf82766725e3",
  "layers": [
    {
      "hash": "sha256:787f47dbeaac926cc8c927a48e8a84fc97e05fbd1c08a4c9c4ddcaddfd87e2f7",
      "uri": "https://registry.access.redhat.com/webassets/docker/content/dist/containers/rhel8/multiarch/appstream/containers/redhat-ubi8-ubi/blobs/sha256:787f47dbeaac926cc8c927a48e8a84fc97e05fbd1c08a4c9c4ddcaddfd87e2f7",
      "headers": {
        "Referer": [
          "https://registry.access.redhat.com/v2/ubi8/ubi/blobs/sha256:787f47dbeaac926cc8c927a48e8a84fc97e05fbd1c08a4c9c4ddcaddfd87e2f7"
        ]
      }
    },
    {
      "hash": "sha256:6a5240d60dc4e1ade0d103badb2a897e8103ebf0ebe08350cd582bbc9671cc7a",
      "uri": "https://registry.access.redhat.com/webassets/docker/content/dist/containers/rhel8/multiarch/appstream/containers/redhat-ubi8-ubi/blobs/sha256:6a5240d60dc4e1ade0d103badb2a897e8103ebf0ebe08350cd582bbc9671cc7a",
      "headers": {
        "Referer": [
          "https://registry.access.redhat.com/v2/ubi8/ubi/blobs/sha256:6a5240d60dc4e1ade0d103badb2a897e8103ebf0ebe08350cd582bbc9671cc7a"
        ]
      }
    }
  ]
}

Clair version: latest commit in development-4.0 branch
Clair core version: latest commit from master branch (github.com/quay/claircore v0.0.26-0.20200626184927-7b7b45a74544)

Unsupported scan results on Google Distroless images

Description of Problem / Feature Request

When I push Google Distroless Images to my Quay registry, I've got Unsupported security scan results. I.e from Debian Java Image

Expected Outcome

Security findings based on CVE for Debian.

Actual Outcome

Unsupported

Environment

Clair version/image: clair-jwt:v3.2.2
Clair client name/version: clair-jwt:v3.2.2
Host OS: Ubuntu 18.04
Kernel (e.g. uname -a): 4.15.0-76-generic
Kubernetes version (use kubectl version): 1.16.8
Helm version (use helm version): 3.1.2
Network/Firewall setup: canal

Additional infos: in
https://github.com/quay/clair/blob/8cdd815ccdab27a2ded0e68740b27444efca8d1e/ext/featurefmt/dpkg/dpkg.go#L41 is a file regex for "var/lib/dpkg/status". It seems Google holds the package information in one file per package in "var/lib/dpkg/status.d"

Maybe it's easy for a Go programmer to add a loop to catch the package information from there :-) Workaround would be an extra build to provide the required status file with information from the status directory.

False positive for jq (CVE-2016-4074)

Description of Problem / Feature Request

Clair v4.0.0-rc.19 detects already fixed vulnerability CVE-2016-4074 in jq 1.6-r0:
ucp-azure-ip-allocator:3.3.0 found jq 1.6-r0 CVE-2016-4074 (fixed: 1.6_rc1-r0)

The reason seems to be that Clair wrongly interprets the version and as the fixed version is a release candidate and the installed version is already the newest version that contains the fix.

Maybe it more reasonable to use in

claircore/alpine/matcher.go

Line 6 in 04cb53c

version "github.com/knqyf263/go-deb-version"

github.com/knqyf263/go-apk-version module, which can resolve and work with alpine rc versions.

Expected Outcome

ucp-azure-ip-allocator:3.3.0 ok

Actual Outcome

ucp-azure-ip-allocator:3.3.0 found jq 1.6-r0 CVE-2016-4074 (fixed: 1.6_rc1-r0)

Environment

Clair version/image: v4.0.0-rc.19
Clair client name/version: clairctl version 1.2.8
Host OS: "Ubuntu 20.04.1 LTS"
Kernel (e.g. uname -a): Linux 5.4.0-52-generic
Kubernetes version (use kubectl version): NA
Network/Firewall setup: NA

rpm: remove tar

The call to the external tar command has become more trouble than it's worth, mostly because there's no way to "clean" permissions, so we end up doing that anyway.

Some bug reports indicate that the rpm scanner is filling up disk space because it's reproducing perimssions that are, frankly, nonsense and preventing removal.

Clair v4 and image layers with mediaType ".tar.gzip" are unknown content-type

Description

After digging I think I found the root of the error I got here:

claircore/internal/indexer/fetcher/fetcher.go

Lines 181 to 206 in 3a4e3d3

    
           switch { 
        
           case ct == "application/gzip": 
        
           	fallthrough 
        
           case strings.HasSuffix(ct, ".tar+gzip"): 
        
           	g, err := gzip.NewReader(br) 
        
           	if err != nil { 
        
           		return err 
        
           	} 
        
           	defer g.Close() 
        
           	r = g 
        
           case ct == "application/zstd": 
        
           	fallthrough 
        
           case strings.HasSuffix(ct, ".tar+zstd"): 
        
           	s, err := zstd.NewReader(br) 
        
           	if err != nil { 
        
           		return err 
        
           	} 
        
           	defer s.Close() 
        
           	r = s 
        
           case ct == "application/x-tar": 
        
           	fallthrough 
        
           case strings.HasSuffix(ct, ".tar"): 
        
           	r = br 
        
           default: 
        
           	return fmt.Errorf("fetcher: unknown content-type %q", ct) 
        
           }

Seems the format "application / vnd.docker.image.rootfs.diff.tar.gzip" is not found in the rules, goes to "default" and return the error

Payload and responses

Image manifest

{
   "schemaVersion": 2,
   "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
   "config": {
      "mediaType": "application/vnd.docker.container.image.v1+json",
      "size": 10077,
      "digest": "sha256:4741a49ceb16bb4a0cf5c6afc259f9a9a437a3bb7cdb15d38a500076c6d43611"
   },
   "layers": [
      {
         "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
         "size": 45344292,
         "digest": "sha256:741437d97401b83849ccbfe4ed8964049b752081647d7f1ea8aea29d989f8968"
      },
      {
         "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
         "size": 10781099,
         "digest": "sha256:34d8874714d74b636739b8a52204650a664fca8ff9741dd66810f30196f103e2"
      },....
   ]
}

Payload to Clair API

{
  "hash": "sha256:e1a7cf003f4b14a70bc057b31a7bc7823663c041e7cbdd8ed99d59de09573b3a",
  "layers": [
    {
      "hash": "sha256:741437d97401b83849ccbfe4ed8964049b752081647d7f1ea8aea29d989f8968",
      "uri": "https://<nexus-repositoty>/v2/<image>/blobs/sha256:741437d97401b83849ccbfe4ed8964049b752081647d7f1ea8aea29d989f8968",
      "headers": {
        "Authorization": [
          "Bearer <token>"
        ]
      }
    }
  ]
}

Stderr

	"level": "error",
	"component": "init/NewHttpTransport",
	"component": "libindex/Libindex.Index",
	"manifest": "sha256:e1a7cf003f4b14a70bc057b31a7bc7823663c041e7cbdd8ed99d59de09573b3a",
	"component": "libindex/Libindex.index",
	"component": "internal/indexer/controller/Controller.Index",
	"manifest": "sha256:e1a7cf003f4b14a70bc057b31a7bc7823663c041e7cbdd8ed99d59de09573b3a",
	"state": "CheckManifest",
	"error": "failed to fetch layers: encountered error while fetching a layer: fetcher: unknown content-type \"application/vnd.docker.image.rootfs.diff.tar.gzip\"",
	"time": "2021-01-28T17:41:53Z",
	"time": "2021-01-28T17:41:53Z",
	"message": "error during scan"
}

Reponse from Clair API

    "manifest_hash": "sha256:e1a7cf003f4b14a70bc057b31a7bc7823663c041e7cbdd8ed99d59de09573b3a",
    "state": "IndexError",
    "packages": {},
    "distributions": {},
    "repository": {},
    "environments": {},
    "success": false,
    "err": "failed to fetch layers: encountered error while fetching a layer: fetcher: unknown content-type \"application/vnd.docker.image.rootfs.diff.tar.gzip\""
}

Cheers
Ederson :)

failed to scan all layer contents: zip error: zip: not a valid zip file

I've seen this message in logs, and I'm not sure where it comes from.

Needs investigation.

Scanning of layers fails for some of the docker image .

Whenever we try to generate report using the cctool , It gives an error of i/o timeout for some of the layers.But after 3-4 days it works fine listing all the vulnerabilities of an image. Can anyone tell why this is happening??

These are the attached screenshots for your reference.

There is now way to limit number of indexer's DB connections

So I recently deployed clair from this commit to quite a big environment. The reason why I mention the commit is that before the deployment, I did not have @hdonnay 's postgres refactor deployed. I don't know if that's significant information, but I think it might given these changes. What happened is that indexer pods used up way more DB connections than I expected. Details of the issue are described in my message to @ldelossa.

I read the code for a bit since then and as far as I can understand, there's no real way to ultimately limit the overall number of how many DB connections one indexer instance creates. Keep in mind that my understanding might be wrong, but let me walk you through it.

First the easy part. One connection pool is created during store initialization. The number of max connections is hard-coded to 30. As described in quay/clair/issues/1179, it'd be nice to have some control over that. But that's not the main problem because we could just scale down the number of indexer pods to deal with that. I think the catch is in different place.

When a POST request is made to index_report endpoint, Index method of Libindex is eventually called. That in turn makes call to controllerFactory function. And each call to that function creates a new connection pool with options acquired from connstring. If an option is not specified in connstring, it falls back to some sane defaults chosen by library authors. For number of maximum connections, it's 4. Furthermore, value of defaultMaxConnIdleTime is 30 minutes. With all this being said, I think it's apparent now that when there's a lot of index requests, one might run out of DB connections.

Now please correct me if I'm wrong in my analysis. But if not, I think we need a mechanism how to limit the overall number of DB connections indexer may make.

changelog entries are all goofed up

See commits like d4a3e8c

java: unidentified jar

Got a report of indexing failures with an unidentified jar error.

I've got an offending jar -- I don't know if it's publicly available.

Clair v4 finds incorrect package version and vulnerabilities in ubutu-based image

Description of Problem / Feature Request

In ubuntu:bionic-based image docker-prod-local.docker.mirantis.net/nc/nagios:4.4.3-bionic-1616603544442 Clair v4 finds incorrect version of installed package libzstd1.

Package version of libzstd1 inside the image is 1.3.3+dfsg-2ubuntu1.2:

root@e721a9d42d9d:/# dpkg -l | grep libzstd1
ii  libzstd1:amd64                1.3.3+dfsg-2ubuntu1.2               amd64        fast lossless compression algorithm

But Clair v4 detects it like libzstd1 (1.3.3+dfsg-2ubuntu1.1) and reports 2 CVE:

nagios:4.4.3-bionic-1616586253497 found libzstd1     1.3.3+dfsg-2ubuntu1.1  CVE-2021-24031 on Ubuntu 18.04 LTS (bionic) - medium. (fixed: 0:1.3.3+dfsg-2ubuntu1.2)
nagios:4.4.3-bionic-1616586253497 found libzstd1     1.3.3+dfsg-2ubuntu1.1  CVE-2021-24032 on Ubuntu 18.04 LTS (bionic) - medium. (fixed: 0:1.3.3+dfsg-2ubuntu1.2)

The whole Clair4 report for docker-prod-local.docker.mirantis.net/nc/nagios:4.4.3-bionic-1616603544442 looks like:

stavrovskaya@stavrovskaya-pc:~/clair/cmd/clairctl$ ./clairctl report docker-prod-local.artifactory.mirantis.com/nc/nagios:4.4.3-bionic-1616586253497
nagios:4.4.3-bionic-1616586253497 found passwd       1:4.5-1ubuntu2         CVE-2013-4235 on Ubuntu 18.04 LTS (bionic) - low.
nagios:4.4.3-bionic-1616586253497 found passwd       1:4.5-1ubuntu2         CVE-2018-7169 on Ubuntu 18.04 LTS (bionic) - low.
nagios:4.4.3-bionic-1616586253497 found ncurses-base 6.1-1ubuntu1.18.04     CVE-2018-19211 on Ubuntu 18.04 LTS (bionic) - low.
nagios:4.4.3-bionic-1616586253497 found ncurses-base 6.1-1ubuntu1.18.04     CVE-2019-17594 on Ubuntu 18.04 LTS (bionic) - negligible.
nagios:4.4.3-bionic-1616586253497 found ncurses-base 6.1-1ubuntu1.18.04     CVE-2019-17595 on Ubuntu 18.04 LTS (bionic) - negligible.
nagios:4.4.3-bionic-1616586253497 found libc-bin     2.27-3ubuntu1.4        CVE-2009-5155 on Ubuntu 18.04 LTS (bionic) - negligible.
nagios:4.4.3-bionic-1616586253497 found libc-bin     2.27-3ubuntu1.4        CVE-2015-8985 on Ubuntu 18.04 LTS (bionic) - negligible.
nagios:4.4.3-bionic-1616586253497 found libc-bin     2.27-3ubuntu1.4        CVE-2016-10228 on Ubuntu 18.04 LTS (bionic) - negligible.
nagios:4.4.3-bionic-1616586253497 found libc-bin     2.27-3ubuntu1.4        CVE-2016-10739 on Ubuntu 18.04 LTS (bionic) - low.
nagios:4.4.3-bionic-1616586253497 found libc-bin     2.27-3ubuntu1.4        CVE-2019-25013 on Ubuntu 18.04 LTS (bionic) - low.
nagios:4.4.3-bionic-1616586253497 found libc-bin     2.27-3ubuntu1.4        CVE-2020-27618 on Ubuntu 18.04 LTS (bionic) - low.
nagios:4.4.3-bionic-1616586253497 found libc-bin     2.27-3ubuntu1.4        CVE-2020-6096 on Ubuntu 18.04 LTS (bionic) - low.
nagios:4.4.3-bionic-1616586253497 found libc-bin     2.27-3ubuntu1.4        CVE-2021-27645 on Ubuntu 18.04 LTS (bionic) - medium.
nagios:4.4.3-bionic-1616586253497 found libc-bin     2.27-3ubuntu1.4        CVE-2021-3326 on Ubuntu 18.04 LTS (bionic) - low.
nagios:4.4.3-bionic-1616586253497 found libc-bin     2.27-3ubuntu1.4        CVE-2018-20796 on Ubuntu 18.04 LTS (bionic) - negligible.
nagios:4.4.3-bionic-1616586253497 found libc-bin     2.27-3ubuntu1.4        CVE-2019-1010022 on Ubuntu 18.04 LTS (bionic) - low.
nagios:4.4.3-bionic-1616586253497 found libc-bin     2.27-3ubuntu1.4        CVE-2019-1010023 on Ubuntu 18.04 LTS (bionic) - low.
nagios:4.4.3-bionic-1616586253497 found libc-bin     2.27-3ubuntu1.4        CVE-2019-1010024 on Ubuntu 18.04 LTS (bionic) - low.
nagios:4.4.3-bionic-1616586253497 found libc-bin     2.27-3ubuntu1.4        CVE-2019-6488 on Ubuntu 18.04 LTS (bionic) - negligible.
nagios:4.4.3-bionic-1616586253497 found libc-bin     2.27-3ubuntu1.4        CVE-2019-7309 on Ubuntu 18.04 LTS (bionic) - negligible.
nagios:4.4.3-bionic-1616586253497 found gcc-8-base   8.4.0-1ubuntu1~18.04   CVE-2018-12886 on Ubuntu 18.04 LTS (bionic) - low.
nagios:4.4.3-bionic-1616586253497 found gcc-8-base   8.4.0-1ubuntu1~18.04   CVE-2019-15847 on Ubuntu 18.04 LTS (bionic) - negligible.
nagios:4.4.3-bionic-1616586253497 found gcc-8-base   8.4.0-1ubuntu1~18.04   CVE-2020-13844 on Ubuntu 18.04 LTS (bionic) - medium.
nagios:4.4.3-bionic-1616586253497 found libtinfo5    6.1-1ubuntu1.18.04     CVE-2018-19211 on Ubuntu 18.04 LTS (bionic) - low.
nagios:4.4.3-bionic-1616586253497 found libtinfo5    6.1-1ubuntu1.18.04     CVE-2019-17594 on Ubuntu 18.04 LTS (bionic) - negligible.
nagios:4.4.3-bionic-1616586253497 found libtinfo5    6.1-1ubuntu1.18.04     CVE-2019-17595 on Ubuntu 18.04 LTS (bionic) - negligible.
nagios:4.4.3-bionic-1616586253497 found libstdc++6   8.4.0-1ubuntu1~18.04   CVE-2018-12886 on Ubuntu 18.04 LTS (bionic) - low.
nagios:4.4.3-bionic-1616586253497 found libstdc++6   8.4.0-1ubuntu1~18.04   CVE-2019-15847 on Ubuntu 18.04 LTS (bionic) - negligible.
nagios:4.4.3-bionic-1616586253497 found libstdc++6   8.4.0-1ubuntu1~18.04   CVE-2020-13844 on Ubuntu 18.04 LTS (bionic) - medium.
nagios:4.4.3-bionic-1616586253497 found login        1:4.5-1ubuntu2         CVE-2013-4235 on Ubuntu 18.04 LTS (bionic) - low.
nagios:4.4.3-bionic-1616586253497 found login        1:4.5-1ubuntu2         CVE-2018-7169 on Ubuntu 18.04 LTS (bionic) - low.
nagios:4.4.3-bionic-1616586253497 found ncurses-bin  6.1-1ubuntu1.18.04     CVE-2018-19211 on Ubuntu 18.04 LTS (bionic) - low.
nagios:4.4.3-bionic-1616586253497 found ncurses-bin  6.1-1ubuntu1.18.04     CVE-2019-17594 on Ubuntu 18.04 LTS (bionic) - negligible.
nagios:4.4.3-bionic-1616586253497 found ncurses-bin  6.1-1ubuntu1.18.04     CVE-2019-17595 on Ubuntu 18.04 LTS (bionic) - negligible.
nagios:4.4.3-bionic-1616586253497 found libgcc1      1:8.4.0-1ubuntu1~18.04 CVE-2018-12886 on Ubuntu 18.04 LTS (bionic) - low.
nagios:4.4.3-bionic-1616586253497 found libgcc1      1:8.4.0-1ubuntu1~18.04 CVE-2019-15847 on Ubuntu 18.04 LTS (bionic) - negligible.
nagios:4.4.3-bionic-1616586253497 found libgcc1      1:8.4.0-1ubuntu1~18.04 CVE-2020-13844 on Ubuntu 18.04 LTS (bionic) - medium.
nagios:4.4.3-bionic-1616586253497 found libudev1     237-3ubuntu10.44       CVE-2018-20839 on Ubuntu 18.04 LTS (bionic) - medium.
nagios:4.4.3-bionic-1616586253497 found libudev1     237-3ubuntu10.44       CVE-2019-9619 on Ubuntu 18.04 LTS (bionic) - low.
nagios:4.4.3-bionic-1616586253497 found libudev1     237-3ubuntu10.44       CVE-2020-13776 on Ubuntu 18.04 LTS (bionic) - low.
nagios:4.4.3-bionic-1616586253497 found requests     2.18.4                 pyup.io-36546 (CVE-2018-18074)
nagios:4.4.3-bionic-1616586253497 found urllib3      1.22                   pyup.io-37055 (CVE-2019-11236)
nagios:4.4.3-bionic-1616586253497 found urllib3      1.22                   pyup.io-36541 (CVE-2018-20060)
nagios:4.4.3-bionic-1616586253497 found urllib3      1.22                   pyup.io-37071 (CVE-2019-11324)
nagios:4.4.3-bionic-1616586253497 found urllib3      1.22                   pyup.io-38834 (CVE-2020-26137)
nagios:4.4.3-bionic-1616586253497 found pip          9.0.1                  pyup.io-38765 (CVE-2019-20916)
nagios:4.4.3-bionic-1616586253497 found libnettle6   3.4-1                  CVE-2018-16869 on Ubuntu 18.04 LTS (bionic) - low.
nagios:4.4.3-bionic-1616586253497 found gpgv         2.2.4-1ubuntu1.4       CVE-2019-13050 on Ubuntu 18.04 LTS (bionic) - low.
nagios:4.4.3-bionic-1616586253497 found liblz4-1     0.0~r131-2ubuntu3      CVE-2019-17543 on Ubuntu 18.04 LTS (bionic) - low.
nagios:4.4.3-bionic-1616586253497 found libgcrypt20  1.8.1-4ubuntu1.2       CVE-2019-12904 on Ubuntu 18.04 LTS (bionic) - low.
nagios:4.4.3-bionic-1616586253497 found libncursesw5 6.1-1ubuntu1.18.04     CVE-2018-19211 on Ubuntu 18.04 LTS (bionic) - low.
nagios:4.4.3-bionic-1616586253497 found libncursesw5 6.1-1ubuntu1.18.04     CVE-2019-17594 on Ubuntu 18.04 LTS (bionic) - negligible.
nagios:4.4.3-bionic-1616586253497 found libncursesw5 6.1-1ubuntu1.18.04     CVE-2019-17595 on Ubuntu 18.04 LTS (bionic) - negligible.
nagios:4.4.3-bionic-1616586253497 found bash         4.4.18-2ubuntu1.2      CVE-2019-18276 on Ubuntu 18.04 LTS (bionic) - low.
nagios:4.4.3-bionic-1616586253497 found libncurses5  6.1-1ubuntu1.18.04     CVE-2018-19211 on Ubuntu 18.04 LTS (bionic) - low.
nagios:4.4.3-bionic-1616586253497 found libncurses5  6.1-1ubuntu1.18.04     CVE-2019-17594 on Ubuntu 18.04 LTS (bionic) - negligible.
nagios:4.4.3-bionic-1616586253497 found libncurses5  6.1-1ubuntu1.18.04     CVE-2019-17595 on Ubuntu 18.04 LTS (bionic) - negligible.
nagios:4.4.3-bionic-1616586253497 found libgnutls30  3.5.18-1ubuntu1.4      CVE-2018-16868 on Ubuntu 18.04 LTS (bionic) - low.
nagios:4.4.3-bionic-1616586253497 found libsystemd0  237-3ubuntu10.44       CVE-2018-20839 on Ubuntu 18.04 LTS (bionic) - medium.
nagios:4.4.3-bionic-1616586253497 found libsystemd0  237-3ubuntu10.44       CVE-2019-9619 on Ubuntu 18.04 LTS (bionic) - low.
nagios:4.4.3-bionic-1616586253497 found libsystemd0  237-3ubuntu10.44       CVE-2020-13776 on Ubuntu 18.04 LTS (bionic) - low.
nagios:4.4.3-bionic-1616586253497 found libzstd1     1.3.3+dfsg-2ubuntu1.1  CVE-2021-24031 on Ubuntu 18.04 LTS (bionic) - medium. (fixed: 0:1.3.3+dfsg-2ubuntu1.2)
nagios:4.4.3-bionic-1616586253497 found libzstd1     1.3.3+dfsg-2ubuntu1.1  CVE-2021-24032 on Ubuntu 18.04 LTS (bionic) - medium. (fixed: 0:1.3.3+dfsg-2ubuntu1.2)
nagios:4.4.3-bionic-1616586253497 found libc6        2.27-3ubuntu1.4        CVE-2009-5155 on Ubuntu 18.04 LTS (bionic) - negligible.
nagios:4.4.3-bionic-1616586253497 found libc6        2.27-3ubuntu1.4        CVE-2015-8985 on Ubuntu 18.04 LTS (bionic) - negligible.
nagios:4.4.3-bionic-1616586253497 found libc6        2.27-3ubuntu1.4        CVE-2016-10228 on Ubuntu 18.04 LTS (bionic) - negligible.
nagios:4.4.3-bionic-1616586253497 found libc6        2.27-3ubuntu1.4        CVE-2016-10739 on Ubuntu 18.04 LTS (bionic) - low.
nagios:4.4.3-bionic-1616586253497 found libc6        2.27-3ubuntu1.4        CVE-2019-25013 on Ubuntu 18.04 LTS (bionic) - low.
nagios:4.4.3-bionic-1616586253497 found libc6        2.27-3ubuntu1.4        CVE-2020-27618 on Ubuntu 18.04 LTS (bionic) - low.
nagios:4.4.3-bionic-1616586253497 found libc6        2.27-3ubuntu1.4        CVE-2020-6096 on Ubuntu 18.04 LTS (bionic) - low.
nagios:4.4.3-bionic-1616586253497 found libc6        2.27-3ubuntu1.4        CVE-2021-27645 on Ubuntu 18.04 LTS (bionic) - medium.
nagios:4.4.3-bionic-1616586253497 found libc6        2.27-3ubuntu1.4        CVE-2021-3326 on Ubuntu 18.04 LTS (bionic) - low.
nagios:4.4.3-bionic-1616586253497 found libc6        2.27-3ubuntu1.4        CVE-2018-20796 on Ubuntu 18.04 LTS (bionic) - negligible.
nagios:4.4.3-bionic-1616586253497 found libc6        2.27-3ubuntu1.4        CVE-2019-1010022 on Ubuntu 18.04 LTS (bionic) - low.
nagios:4.4.3-bionic-1616586253497 found libc6        2.27-3ubuntu1.4        CVE-2019-1010023 on Ubuntu 18.04 LTS (bionic) - low.
nagios:4.4.3-bionic-1616586253497 found libc6        2.27-3ubuntu1.4        CVE-2019-1010024 on Ubuntu 18.04 LTS (bionic) - low.
nagios:4.4.3-bionic-1616586253497 found libc6        2.27-3ubuntu1.4        CVE-2019-6488 on Ubuntu 18.04 LTS (bionic) - negligible.
nagios:4.4.3-bionic-1616586253497 found libc6        2.27-3ubuntu1.4        CVE-2019-7309 on Ubuntu 18.04 LTS (bionic) - negligible.
nagios:4.4.3-bionic-1616586253497 found libpcre3     2:8.39-9               CVE-2017-11164 on Ubuntu 18.04 LTS (bionic) - negligible.
nagios:4.4.3-bionic-1616586253497 found libpcre3     2:8.39-9               CVE-2019-20838 on Ubuntu 18.04 LTS (bionic) - low.
nagios:4.4.3-bionic-1616586253497 found libpcre3     2:8.39-9               CVE-2020-14155 on Ubuntu 18.04 LTS (bionic) - negligible.
nagios:4.4.3-bionic-1616586253497 found coreutils    8.28-1ubuntu1          CVE-2016-2781 on Ubuntu 18.04 LTS (bionic) - low.
nagios:4.4.3-bionic-1616586253497 found coreutils    8.28-1ubuntu1          CVE-2017-18018 on Ubuntu 18.04 LTS (bionic) - low.
nagios:4.4.3-bionic-1616586253497 found tar          1.29b-2ubuntu0.2       CVE-2021-20193 on Ubuntu 18.04 LTS (bionic) - low.
nagios:4.4.3-bionic-1616586253497 found libtasn1-6   4.13-2                 CVE-2018-1000654 on Ubuntu 18.04 LTS (bionic) - negligible.
nagios:4.4.3-bionic-1616586253497 found libhogweed4  3.4-1                  CVE-2018-16869 on Ubuntu 18.04 LTS (bionic) - low.

Environment

Clair version/image: v4.0
Clair client name/version:
Host OS: ubuntu:bionic
Kernel (e.g. uname -a):
Kubernetes version (use kubectl version):
Network/Firewall setup:

Add support for modular rpm

Modular rpms are a special type of rpms which require different vulnerability detection.

Modular rpms are stored in the rpm database and can be distinguished from normal rpms by RPMTAG_MODULARITYLABEL label.

Clair v3 already has a modular rpm detection: quay/clair#891

In oval vulnerability data, modular rpms can be also distinguished by looking at criterion comments.

<criterion test_ref="oval:com.redhat.rhsa:tst:20190980027" comment="Module httpd:2.4 is enabled"/>

Vulnerability Version Operation Comparison

The vulnerability data model should grow a constant string tag indicating how to compare a package's version's to a vulnerability.

OVAL databases state objects define "operation" keys which inform what direction a version comparison indicates vulnerability.

const (
	_ Operation = iota // Invalid

	OpEquals                   // equals
	OpNotEquals                // not equals
	OpCaseInsensitiveEquals    // case insensitive equals
	OpCaseInsensitiveNotEquals // case insensitive not equals
	OpGreaterThan              // greater than
	OpLessThan                 // less than
	OpGreaterThanOrEqual       // greater than or equal
	OpLessThanOrEqual          // less than or equal
	OpBitwiseAnd               // bitwise and
	OpBitwiseOr                // bitwise or
	OpPatternMatch             // pattern match
	OpSubset                   // subset of
	OpSuperset                 // superset of
)

The Vulnerability model should gain a field to express at least he most common EVR related ones

        OpGreaterThan                    // greater than
	OpLessThan                         // less than
	OpGreaterThanOrEqual       // greater than or equal
	OpLessThanOrEqual          // less than or equal

Matcher implementations can then utilize the Vulnerability field in their version checks.

Clair v4 creates CVEs duplicates in database

Description of Problem / Feature Request

In ubuntu:bionic image Clair v4 finds CVE duplicates in package libzstd1:

$  ./clairctl report ubuntu:bionic
ubuntu:bionic found passwd       1:4.5-1ubuntu2         CVE-2013-4235 on Ubuntu 18.04 LTS (bionic) - low.
ubuntu:bionic found passwd       1:4.5-1ubuntu2         CVE-2018-7169 on Ubuntu 18.04 LTS (bionic) - low.
ubuntu:bionic found ncurses-base 6.1-1ubuntu1.18.04     CVE-2018-19211 on Ubuntu 18.04 LTS (bionic) - low.
ubuntu:bionic found ncurses-base 6.1-1ubuntu1.18.04     CVE-2019-17594 on Ubuntu 18.04 LTS (bionic) - negligible.
ubuntu:bionic found ncurses-base 6.1-1ubuntu1.18.04     CVE-2019-17595 on Ubuntu 18.04 LTS (bionic) - negligible.
ubuntu:bionic found libc-bin     2.27-3ubuntu1.4        CVE-2009-5155 on Ubuntu 18.04 LTS (bionic) - negligible.
ubuntu:bionic found libc-bin     2.27-3ubuntu1.4        CVE-2015-8985 on Ubuntu 18.04 LTS (bionic) - negligible.
ubuntu:bionic found libc-bin     2.27-3ubuntu1.4        CVE-2016-10228 on Ubuntu 18.04 LTS (bionic) - negligible.
ubuntu:bionic found libc-bin     2.27-3ubuntu1.4        CVE-2016-10739 on Ubuntu 18.04 LTS (bionic) - low.
ubuntu:bionic found libc-bin     2.27-3ubuntu1.4        CVE-2019-25013 on Ubuntu 18.04 LTS (bionic) - low.
ubuntu:bionic found libc-bin     2.27-3ubuntu1.4        CVE-2020-27618 on Ubuntu 18.04 LTS (bionic) - low.
ubuntu:bionic found libc-bin     2.27-3ubuntu1.4        CVE-2020-6096 on Ubuntu 18.04 LTS (bionic) - low.
ubuntu:bionic found libc-bin     2.27-3ubuntu1.4        CVE-2021-3326 on Ubuntu 18.04 LTS (bionic) - low.
ubuntu:bionic found libc-bin     2.27-3ubuntu1.4        CVE-2018-20796 on Ubuntu 18.04 LTS (bionic) - negligible.
ubuntu:bionic found libc-bin     2.27-3ubuntu1.4        CVE-2019-1010022 on Ubuntu 18.04 LTS (bionic) - low.
ubuntu:bionic found libc-bin     2.27-3ubuntu1.4        CVE-2019-1010023 on Ubuntu 18.04 LTS (bionic) - low.
ubuntu:bionic found libc-bin     2.27-3ubuntu1.4        CVE-2019-1010024 on Ubuntu 18.04 LTS (bionic) - low.
ubuntu:bionic found libc-bin     2.27-3ubuntu1.4        CVE-2019-6488 on Ubuntu 18.04 LTS (bionic) - negligible.
ubuntu:bionic found libc-bin     2.27-3ubuntu1.4        CVE-2019-7309 on Ubuntu 18.04 LTS (bionic) - negligible.
ubuntu:bionic found libc-bin     2.27-3ubuntu1.4        CVE-2021-27645 on Ubuntu 18.04 LTS (bionic) - medium.
ubuntu:bionic found libc-bin     2.27-3ubuntu1.4        CVE-2020-27618 on Ubuntu 18.04 LTS (bionic) - low.
ubuntu:bionic found gcc-8-base   8.4.0-1ubuntu1~18.04   CVE-2018-12886 on Ubuntu 18.04 LTS (bionic) - low.
ubuntu:bionic found gcc-8-base   8.4.0-1ubuntu1~18.04   CVE-2019-15847 on Ubuntu 18.04 LTS (bionic) - negligible.
ubuntu:bionic found gcc-8-base   8.4.0-1ubuntu1~18.04   CVE-2020-13844 on Ubuntu 18.04 LTS (bionic) - medium.
ubuntu:bionic found libtinfo5    6.1-1ubuntu1.18.04     CVE-2018-19211 on Ubuntu 18.04 LTS (bionic) - low.
ubuntu:bionic found libtinfo5    6.1-1ubuntu1.18.04     CVE-2019-17594 on Ubuntu 18.04 LTS (bionic) - negligible.
ubuntu:bionic found libtinfo5    6.1-1ubuntu1.18.04     CVE-2019-17595 on Ubuntu 18.04 LTS (bionic) - negligible.
ubuntu:bionic found libstdc++6   8.4.0-1ubuntu1~18.04   CVE-2018-12886 on Ubuntu 18.04 LTS (bionic) - low.
ubuntu:bionic found libstdc++6   8.4.0-1ubuntu1~18.04   CVE-2019-15847 on Ubuntu 18.04 LTS (bionic) - negligible.
ubuntu:bionic found libstdc++6   8.4.0-1ubuntu1~18.04   CVE-2020-13844 on Ubuntu 18.04 LTS (bionic) - medium.
ubuntu:bionic found gpgv         2.2.4-1ubuntu1.4       CVE-2019-13050 on Ubuntu 18.04 LTS (bionic) - low.
ubuntu:bionic found login        1:4.5-1ubuntu2         CVE-2013-4235 on Ubuntu 18.04 LTS (bionic) - low.
ubuntu:bionic found login        1:4.5-1ubuntu2         CVE-2018-7169 on Ubuntu 18.04 LTS (bionic) - low.
ubuntu:bionic found ncurses-bin  6.1-1ubuntu1.18.04     CVE-2018-19211 on Ubuntu 18.04 LTS (bionic) - low.
ubuntu:bionic found ncurses-bin  6.1-1ubuntu1.18.04     CVE-2019-17594 on Ubuntu 18.04 LTS (bionic) - negligible.
ubuntu:bionic found ncurses-bin  6.1-1ubuntu1.18.04     CVE-2019-17595 on Ubuntu 18.04 LTS (bionic) - negligible.
ubuntu:bionic found libgcc1      1:8.4.0-1ubuntu1~18.04 CVE-2018-12886 on Ubuntu 18.04 LTS (bionic) - low.
ubuntu:bionic found libgcc1      1:8.4.0-1ubuntu1~18.04 CVE-2019-15847 on Ubuntu 18.04 LTS (bionic) - negligible.
ubuntu:bionic found libgcc1      1:8.4.0-1ubuntu1~18.04 CVE-2020-13844 on Ubuntu 18.04 LTS (bionic) - medium.
ubuntu:bionic found libudev1     237-3ubuntu10.44       CVE-2018-20839 on Ubuntu 18.04 LTS (bionic) - medium.
ubuntu:bionic found libudev1     237-3ubuntu10.44       CVE-2019-9619 on Ubuntu 18.04 LTS (bionic) - low.
ubuntu:bionic found libudev1     237-3ubuntu10.44       CVE-2020-13776 on Ubuntu 18.04 LTS (bionic) - low.
ubuntu:bionic found libnettle6   3.4-1                  CVE-2018-16869 on Ubuntu 18.04 LTS (bionic) - low.
ubuntu:bionic found liblz4-1     0.0~r131-2ubuntu3      CVE-2019-17543 on Ubuntu 18.04 LTS (bionic) - low.
ubuntu:bionic found libgcrypt20  1.8.1-4ubuntu1.2       CVE-2019-12904 on Ubuntu 18.04 LTS (bionic) - low.
ubuntu:bionic found libncursesw5 6.1-1ubuntu1.18.04     CVE-2018-19211 on Ubuntu 18.04 LTS (bionic) - low.
ubuntu:bionic found libncursesw5 6.1-1ubuntu1.18.04     CVE-2019-17594 on Ubuntu 18.04 LTS (bionic) - negligible.
ubuntu:bionic found libncursesw5 6.1-1ubuntu1.18.04     CVE-2019-17595 on Ubuntu 18.04 LTS (bionic) - negligible.
ubuntu:bionic found bash         4.4.18-2ubuntu1.2      CVE-2019-18276 on Ubuntu 18.04 LTS (bionic) - low.
ubuntu:bionic found libncurses5  6.1-1ubuntu1.18.04     CVE-2018-19211 on Ubuntu 18.04 LTS (bionic) - low.
ubuntu:bionic found libncurses5  6.1-1ubuntu1.18.04     CVE-2019-17594 on Ubuntu 18.04 LTS (bionic) - negligible.
ubuntu:bionic found libncurses5  6.1-1ubuntu1.18.04     CVE-2019-17595 on Ubuntu 18.04 LTS (bionic) - negligible.
ubuntu:bionic found libgnutls30  3.5.18-1ubuntu1.4      CVE-2018-16868 on Ubuntu 18.04 LTS (bionic) - low.
ubuntu:bionic found libgnutls30  3.5.18-1ubuntu1.4      CVE-2021-20231 on Ubuntu 18.04 LTS (bionic) - medium.
ubuntu:bionic found libgnutls30  3.5.18-1ubuntu1.4      CVE-2021-20232 on Ubuntu 18.04 LTS (bionic) - medium.
ubuntu:bionic found libsystemd0  237-3ubuntu10.44       CVE-2018-20839 on Ubuntu 18.04 LTS (bionic) - medium.
ubuntu:bionic found libsystemd0  237-3ubuntu10.44       CVE-2019-9619 on Ubuntu 18.04 LTS (bionic) - low.
ubuntu:bionic found libsystemd0  237-3ubuntu10.44       CVE-2020-13776 on Ubuntu 18.04 LTS (bionic) - low.
ubuntu:bionic found libzstd1     1.3.3+dfsg-2ubuntu1.1  CVE-2021-24031 on Ubuntu 18.04 LTS (bionic) - medium.
ubuntu:bionic found libzstd1     1.3.3+dfsg-2ubuntu1.1  CVE-2021-24032 on Ubuntu 18.04 LTS (bionic) - medium.
ubuntu:bionic found libzstd1     1.3.3+dfsg-2ubuntu1.1  CVE-2021-24031 on Ubuntu 18.04 LTS (bionic) - medium. (fixed: 0:1.3.3+dfsg-2ubuntu1.2)
ubuntu:bionic found libzstd1     1.3.3+dfsg-2ubuntu1.1  CVE-2021-24032 on Ubuntu 18.04 LTS (bionic) - medium. (fixed: 0:1.3.3+dfsg-2ubuntu1.2)
ubuntu:bionic found libzstd1     1.3.3+dfsg-2ubuntu1.1  CVE-2021-24031 on Ubuntu 18.04 LTS (bionic) - medium. (fixed: 0:1.3.3+dfsg-2ubuntu1.2)
ubuntu:bionic found libzstd1     1.3.3+dfsg-2ubuntu1.1  CVE-2021-24032 on Ubuntu 18.04 LTS (bionic) - medium. (fixed: 0:1.3.3+dfsg-2ubuntu1.2)
ubuntu:bionic found libc6        2.27-3ubuntu1.4        CVE-2009-5155 on Ubuntu 18.04 LTS (bionic) - negligible.
ubuntu:bionic found libc6        2.27-3ubuntu1.4        CVE-2015-8985 on Ubuntu 18.04 LTS (bionic) - negligible.
ubuntu:bionic found libc6        2.27-3ubuntu1.4        CVE-2016-10228 on Ubuntu 18.04 LTS (bionic) - negligible.
ubuntu:bionic found libc6        2.27-3ubuntu1.4        CVE-2016-10739 on Ubuntu 18.04 LTS (bionic) - low.
ubuntu:bionic found libc6        2.27-3ubuntu1.4        CVE-2019-25013 on Ubuntu 18.04 LTS (bionic) - low.
ubuntu:bionic found libc6        2.27-3ubuntu1.4        CVE-2020-27618 on Ubuntu 18.04 LTS (bionic) - low.
ubuntu:bionic found libc6        2.27-3ubuntu1.4        CVE-2020-6096 on Ubuntu 18.04 LTS (bionic) - low.
ubuntu:bionic found libc6        2.27-3ubuntu1.4        CVE-2021-3326 on Ubuntu 18.04 LTS (bionic) - low.
ubuntu:bionic found libc6        2.27-3ubuntu1.4        CVE-2018-20796 on Ubuntu 18.04 LTS (bionic) - negligible.
ubuntu:bionic found libc6        2.27-3ubuntu1.4        CVE-2019-1010022 on Ubuntu 18.04 LTS (bionic) - low.
ubuntu:bionic found libc6        2.27-3ubuntu1.4        CVE-2019-1010023 on Ubuntu 18.04 LTS (bionic) - low.
ubuntu:bionic found libc6        2.27-3ubuntu1.4        CVE-2019-1010024 on Ubuntu 18.04 LTS (bionic) - low.
ubuntu:bionic found libc6        2.27-3ubuntu1.4        CVE-2019-6488 on Ubuntu 18.04 LTS (bionic) - negligible.
ubuntu:bionic found libc6        2.27-3ubuntu1.4        CVE-2019-7309 on Ubuntu 18.04 LTS (bionic) - negligible.
ubuntu:bionic found libc6        2.27-3ubuntu1.4        CVE-2021-27645 on Ubuntu 18.04 LTS (bionic) - medium.
ubuntu:bionic found libc6        2.27-3ubuntu1.4        CVE-2020-27618 on Ubuntu 18.04 LTS (bionic) - low.
ubuntu:bionic found libpcre3     2:8.39-9               CVE-2017-11164 on Ubuntu 18.04 LTS (bionic) - negligible.
ubuntu:bionic found libpcre3     2:8.39-9               CVE-2019-20838 on Ubuntu 18.04 LTS (bionic) - low.
ubuntu:bionic found libpcre3     2:8.39-9               CVE-2020-14155 on Ubuntu 18.04 LTS (bionic) - negligible.
ubuntu:bionic found coreutils    8.28-1ubuntu1          CVE-2016-2781 on Ubuntu 18.04 LTS (bionic) - low.
ubuntu:bionic found coreutils    8.28-1ubuntu1          CVE-2017-18018 on Ubuntu 18.04 LTS (bionic) - low.
ubuntu:bionic found tar          1.29b-2ubuntu0.2       CVE-2021-20193 on Ubuntu 18.04 LTS (bionic) - low.
ubuntu:bionic found libtasn1-6   4.13-2                 CVE-2018-1000654 on Ubuntu 18.04 LTS (bionic) - negligible.
ubuntu:bionic found libhogweed4  3.4-1                  CVE-2018-16869 on Ubuntu 18.04 LTS (bionic) - low.

This CVE duplicates are presented in database with different IDs and have differences in fields: fixed_in_version and description:

[ RECORD 3 ]----------+----------------------------------------------------------------------------------------------------------------------------------
id                     | 168079564
hash_kind              | md5
hash                   | \x12c86ca1844458d93764b733beb604d6
updater                | ubuntu-bionic-updater
name                   | CVE-2021-24031 on Ubuntu 18.04 LTS (bionic) - medium.
description            | zstd adds read permissions to files while being compressed or uncompressed
issued                 | 0001-01-01 00:00:00+00
links                  | https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-24031 http://people.canonical.com/~ubuntu-security/cve/2021/CVE-2021-24031.html http://bugs.debian.org/cgi-bi
n/bugreport.cgi?bug=981404 https://github.com/facebook/zstd/issues/1630
severity               |
normalized_severity    | Medium
package_name           | libzstd1
package_version        |
package_module         |
package_arch           |
package_kind           | binary
dist_id                | ubuntu
dist_name              | Ubuntu
dist_version           | 18.04.3 LTS (Bionic Beaver)
dist_version_code_name | bionic
dist_version_id        | 18.04
dist_arch              |
dist_cpe               |
dist_pretty_name       | Ubuntu 18.04.3 LTS
repo_name              |
repo_key               |
repo_uri               |
fixed_in_version       |
arch_operation         | invalid
vulnerable_range       | empty
version_kind           |
-[ RECORD 11 ]---------+----------------------------------------------------------------------------------------------------------------------------------
id                     | 229026487
hash_kind              | md5
hash                   | \x483176d55a10232efd722a7a3bd1523b
updater                | ubuntu-bionic-updater
name                   | CVE-2021-24031 on Ubuntu 18.04 LTS (bionic) - medium.
description            | zstd adds read permissions to files while being compressed or uncompressed
issued                 | 0001-01-01 00:00:00+00
links                  | https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-24031 http://people.canonical.com/~ubuntu-security/cve/2021/CVE-2021-24031.html https://usn.ubuntu.com/usn/us
n-4760-1 http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=981404 https://github.com/facebook/zstd/issues/1630
severity               |
normalized_severity    | Medium
package_name           | libzstd1
package_version        |
package_module         |
package_arch           |
package_kind           | binary
dist_id                | ubuntu
dist_name              | Ubuntu
dist_version           | 18.04.3 LTS (Bionic Beaver)
dist_version_code_name | bionic
dist_version_id        | 18.04
dist_arch              |
dist_cpe               |
dist_pretty_name       | Ubuntu 18.04.3 LTS
repo_name              |
repo_key               |
repo_uri               |
fixed_in_version       | 0:1.3.3+dfsg-2ubuntu1.2
arch_operation         | invalid
vulnerable_range       | empty
version_kind           |
-[ RECORD 17 ]---------+----------------------------------------------------------------------------------------------------------------------------------
id                     | 244074097
hash_kind              | md5
hash                   | \x6e56deaf1e258356e6d52a18e7f4e58f
updater                | ubuntu-bionic-updater
name                   | CVE-2021-24031 on Ubuntu 18.04 LTS (bionic) - medium.
description            | In the Zstandard command-line utility prior to v1.4.1, output files were created with default permissions. Correct file permissions (matching the input) would only b
e set at completion time. Output files could therefore be readable or writable to unintended parties.
issued                 | 0001-01-01 00:00:00+00
links                  | https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-24031 http://people.canonical.com/~ubuntu-security/cve/2021/CVE-2021-24031.html https://usn.ubuntu.com/usn/us
n-4760-1 http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=981404 https://github.com/facebook/zstd/issues/1630
severity               |
normalized_severity    | Medium
package_name           | libzstd1
package_version        |
package_module         |
package_arch           |
package_kind           | binary
dist_id                | ubuntu
dist_name              | Ubuntu
dist_version           | 18.04.3 LTS (Bionic Beaver)
dist_version_code_name | bionic
dist_version_id        | 18.04
dist_arch              |
dist_cpe               |
dist_pretty_name       | Ubuntu 18.04.3 LTS
repo_name              |
repo_key               |
repo_uri               |
fixed_in_version       | 0:1.3.3+dfsg-2ubuntu1.2
arch_operation         | invalid
vulnerable_range       | empty
version_kind           |

Environment

Clair version/image: v4.0
Clair client name/version:
Host OS: ubuntu:bionic
Kernel (e.g. uname -a):
Kubernetes version (use kubectl version):
Network/Firewall setup:

Add a means to force re-indexing of a manifest.

This would be incredibly useful when developing new scanners. As is, we unconditionally cache results, which is certainly the correct choice in a "real" environment.

	switch {
	case ct == "application/gzip":
	fallthrough
	case strings.HasSuffix(ct, ".tar+gzip"):
	g, err := gzip.NewReader(br)
	if err != nil {
	return err
	}
	defer g.Close()
	r = g
	case ct == "application/zstd":
	fallthrough
	case strings.HasSuffix(ct, ".tar+zstd"):
	s, err := zstd.NewReader(br)
	if err != nil {
	return err
	}
	defer s.Close()
	r = s
	case ct == "application/x-tar":
	fallthrough
	case strings.HasSuffix(ct, ".tar"):
	r = br
	default:
	return fmt.Errorf("fetcher: unknown content-type %q", ct)
	}

quay / claircore Goto Github PK

claircore's Introduction

ClairCore

Local development and testing

claircore's People

Contributors

Stargazers

Watchers

Forkers

claircore's Issues

Description

Description

Payload and responses

Description of Problem / Feature Request

Description of Problem / Feature Request

Expected Outcome

Actual Outcome

Environment

Description of Problem / Feature Request

Expected Outcome

Actual Outcome

Investigation

Environment

Description

Description of Problem / Feature Request

Expected Outcome

Actual Outcome

Environment

Description of Problem / Feature Request

Expected Outcome

Actual Outcome

Environment

Description

Payload and responses

Recommend Projects

Recommend Topics

Recommend Org