arminc / clair-scanner Goto Github PK

View Code? Open in Web Editor NEW

843.0 843.0 151.0 107 KB

Docker containers vulnerability scan

License: Apache License 2.0

Go 83.76% Makefile 5.44% Dockerfile 2.15% Shell 8.64%

docker vulnerability-scanners

clair-scanner's People

Contributors

Stargazers

Watchers

Forkers

bakahr pombredanne commjoen ryarnyah slhad linearregression burrbd evalle anthu mrorgues plasticine dzaporozhets pixelbloom ashokdevop dhirenshumsher slitayem 0x13337 lstefaniszyn cplee usr42 chubbymaggie opt9 leucos ehotinger sanwenkit rakito sheela-nistala antonidabek srivasthav89 luisico cji popawu 3453-315h 5up3rc batermj aubcor01 laurenttardif lanzsec maloney20 anonymouse64 sekhar435 karthiknannapaneni gravis kumarchatla arunnagath pavittr sgcclh aubm tisc0 skjolber exenin shdeanwang atul7cloudyuga nrathaus devopssourav lwsanty rothandrew vithu95 ssprasad100 rafa-pizzi nickryand rakou deepak16686 frantallukas10 pa1kumarbh orlando-jamie ethiansjh miksonx valdemarceccon rzandonai vosscodes l1ahim iandrewdunn abenab modulexcite 5l1v3r1 longfeide2008 latkes greenmind-sec apollusehs-oss meterianhq k1m0n3 romaincabassot chaholl tacsihan projectzerodays sealmindset guddukhan1987 marciopocebon flyforfreedom ali-parsfar silentsoul04 goober voronaam lcarcaramo rei88 santoshbachar clix-dev-llc davar-playgrounds devil865

clair-scanner's Issues

Proposition: add the "fixedby" field in the output table

Hello,
In the output table printed when we run clair-scanner, we find most of the following fields that we also find in the json logs, I think it would be great to also add the field "fixedby" that we find in the logs but not on the table, in order for the user to know right away if a vulnerability present in the output table has been patched and if there is a fix.
It gives a direct idea about where actions can be taken to improve the scan results.

Example:
{
"featurename": "glibc",
"featureversion": "2.24-11+deb9u3",
"vulnerability": "CVE-2017-16997",
"namespace": "debian:9",
"description": "elf/dl-load.c in the GNU C Library (aka glibc or libc6) 2.19 through 2.26 mishandles RPATH and RUNPATH containing $ORIGIN for a privileged (setuid or AT_SECURE) program, which allows local users to gain
privileges via a Trojan horse library in the current working directory, related to the fillin_rpath and decompose_rpath functions. This is associated with misinterpretion of an empty RPATH/RUNPATH token as the "./" directory. NOT
E: this configuration of RPATH/RUNPATH for a privileged program is apparently very uncommon; most likely, no such program is shipped with any common Linux distribution.",
"link": "https://security-tracker.debian.org/tracker/CVE-2017-16997",
"severity": "High",
"fixedby": ""
},
Thanks,

POST to Clair failed to Post

I'm trying to scan an image with clair-scanner and I get the following error:

$ clair-scanner -c http://{{ clair_url }} alpine:3.5
2018/01/25 10:54:48 [INFO] ▶ Start clair-scanner
2018/01/25 10:54:48 [INFO] ▶ Server listening on port 9279
2018/01/25 10:54:48 [INFO] ▶ Analyzing a53e65bf86af4e96e87dc34c40d99d8b25676fb6ccf2c10ec2d1d286c24877b7
2018/01/25 10:55:18 [CRIT] ▶ Could not analyze layer: POST to Clair failed Post http://{{ clair_url }}: dial tcp {{ clair_server_ip }}:80: i/o timeout

The server is listening in the default clair ports 6060 and 6061, and not 80. I've tried
setting the clair_url as http://{{ clair_url }}:6060 or http://{{ clair_url }}:6061 but it doesn't work either.

klar works flawlessly

$ CLAIR_ADDR=http://{{ clair_url }} klar alpine:3.5
Analysing 1 layers
Got results from Clair API v1
Found 0 vulnerabilities

Missing artefacts for v9

Hi,

i'm missing prebuild artefacts for version 9.

https://github.com/arminc/clair-scanner/releases

Issues when scanning as part of Jenkins Pipeline job

Hi folks, I'm having some odd issues with incorporating this into a Pipeline job.

When I run a scan on the Jenkins instance's command line, it completes with no problems, but when I put the scan command into a build step, it scans some of the layers before throwing a 400 error - this is on the same image. The Jenkins instance itself is in a container and is working normally and the Jenkins user has access to the host's Docker socket file (Docker in Docker.)

Does anyone have any ideas what might be causing this? It's the exact same command, only used in two different places - could it be an environment variable or something?

Running one of the examples fails

user@host /tmp $ ./clair-scanner_linux_amd64 nginx:1.11.6-alpine example-nginx.yaml http://127.0.0.1:6060 127.0.0.1
2017-09-19 11:03:45.952450 I | Analyzing 23f9039742dcfc7436c234a037aa1890b36b0a259e009687fe427aeb1691cd22
2017-09-19 11:03:45.954962 I | Analyzing faild: Could not analyze layer: Post http://127.0.0.1:6060/v1/layers: read tcp 127.0.0.1:58900->127.0.0.1:6060: read: connection reset by peer

Make the source gogland compatible

Hi @arminc,
can we make the source Gogland compatible so that it becomes easier to provide contributions as a Go-beginner ;-) ?

Regards,
Jeroen

error "reference does not exist"

I've setup clair and clair_postgres containers and downloaded the latest version of the clair-scanner CLI. It won't work though and I don't understand the error message.

root@build:/srv/docker-compose# clair-scanner_linux_amd64 debian:latest
2019/02/14 17:10:47 [INFO] ▶ Start clair-scanner
2019/02/14 17:10:47 [CRIT] ▶ Could not save Docker image [debian:latest]: Error response from daemon: reference does not exist

root@build:/srv/docker-compose# docker ps
CONTAINER ID        IMAGE                          COMMAND                  CREATED             STATUS              PORTS                              NAMES
c4c9e84e4f37        quay.io/coreos/clair:v2.0.7    "/clair -config /con…"   50 seconds ago      Up 48 seconds       0.0.0.0:6060-6061->6060-6061/tcp   clair
5fa90369d9b3        postgres:11.1                  "docker-entrypoint.s…"   22 minutes ago      Up 22 minutes       5432/tcp                           clair_postgres

Using strace I saw that the communication was through /var/run/docker.sock and the error message part of a 404 response from the server.

read(3, "HTTP/1.1 404 Not Found\r\nApi-Version: 1.39\r\nContent-Type: application/json\r\nDocker-Experimental: false\r\nOstype: linux\r\nServer: Docker/18.09.2 (linux)\r\nDate: Thu, 14 Feb 2019 16:06:27 GMT\r\nContent-Length: 39\r\n\r\n{\"message\":\"reference does not exist\"}\n", 4096) = 248

Or maybe it comes from dockerd itself (notice the "Server: Docker") and not the clair damon inside the Docker container? But what does it mean?

A docker logs clair returns only INFO level messages, the last ones are:

{"Event":"finished fetching","Level":"info","Location":"updater.go:242","Time":"2019-02-14 16:25:24.134101","updater name":"debian"}
{"Event":"finished fetching","Level":"info","Location":"updater.go:242","Time":"2019-02-14 16:25:24.371419","updater name":"alpine"}

List Approved vulnerabilities in the JSON report

The vulnerabilities that are approved are currently in the report, but they are hard to find. Would it be possible to add an "approved": [] section to the report?

Thanks

JSON format not valid on Cucumber reporting plugin

Hi!

Just wanna query that if there are plans to allow the report output in a Cumber-valid format?

Thanks!

Readme should explain public IP requirements

I believe that the process for scanning with clair-scanner is something like

clair-scanner starts a local registry
clair-scanner calls clair requesting a scan using the --IP parameter
clair calls back to the public IP sent and pulls the image

I think this should really be called out in the readme as a major decision in determining if you should use the tool. I was evaluating this for use with my CI build tool but would not be able to open public access back to the build agent.

I can create a PR to update the readme for this if my understanding is correct.

Remove github.com/fatih/color

Add integration tests

Create an integration test that does the following

Create a docker image
Start clair db
Start clair
Run integration test

feat(): supports distroless images

Hi,

Currently, when I try to scan images relying on Distroless (java in my case, but this happen in every distroless base image), the clair-scanner respond with the following element:

 clair-scanner -c http://docker:6060 --ip $(hostname -i) -r gl-container-scanning-report.json -l clair.log -w clair-whitelist.yml ${CI_APPLICATION_REPOSITORY}:${CI_COMMIT_SHA}                                                                                                                      ║
  ║ 2019/01/21 10:38:31  ▶ Start clair-scanner                                                                                                                                                                                                                                                            ║
  ║ 2019/01/21 10:38:33  ▶ Server listening on port 9279                                                                                                                                                                                                                                                  ║
  ║ 2019/01/21 10:38:33  ▶ Analyzing 2bd5d818a479debbbf49c31a7fd62cdaca7490ca3412c531b193ce9e2e342e5a                                                                                                                                                                                                     ║
  ║ 2019/01/21 10:38:33  ▶ Analyzing 6f2c029817652d184bffa480a2dc32c1a01c35747e4a1278a6064f18e92e3917                                                                                                                                                                                                     ║
  ║ 2019/01/21 10:38:33  ▶ Analyzing 41caffa90d5a283dac94f170ccd82c7e23a17532392eea388627fb8700a352ae                                                                                                                                                                                                     ║
  ║ 2019/01/21 10:38:33  ▶ Analyzing 74d2c2ab8fd47f8864c20d790950a67a95f793b934a1c986e18c7238138fa37c                                                                                                                                                                                                     ║
  ║ 2019/01/21 10:38:33  ▶ Analyzing 556cb9d1211301811fcc9a0f2b2e1b483b570381da1863cd179fd1e321ce6852                                                                                                                                                                                                     ║
  ║ 2019/01/21 10:38:33  ▶ Analyzing 800d390ab06866c24814a6a283bed53bb63d908724995688fa0971ee6c7ecabb                                                                                                                                                                                                     ║
  ║ 2019/01/21 10:38:33  ▶ Could not fetch vulnerabilities. No features have been detected in the image. This usually means that the image isn't supported by Clair

The problem is the program ends up on error (exit code 1) due to this message and potentially breaks our CI pipeline.

This is caused by this line :

clair-scanner/clair.go

Line 84 in 871c1de

    
           logger.Fatal("Could not fetch vulnerabilities. No features have been detected in the image. This usually means that the image isn't supported by Clair")

I think the program shouldn't end on error in such case, because we don't have any distribution nor flaws in our container at the clair level.

Do you have a solution about this problem ? From my point of view, we could have a flag to allow this error to be Fatal or not.

Thanks

/cc @Neonox31

Add coverage status with coveralls.io

[CRIT] ▶ Could not analyze layer: Clair responded with a failure: Got response 400 with message {"Error":{"Message":"could not find layer"}}

Hi,

I'm using a docker container with the clair-scanner installed and the docker daemon of the host available to it, so that it can access the images and containers of the host.

So in order to do a scan for example I'm using the below:
docker run -it --rm -v /var/run/docker.sock:/var/run/docker.sock -v /lib64/libdevmapper.so.1.02:/usr/lib/x86_64-linux-gnu/libdevmapper.so.1.02 -v /lib64/libudev.so.0:/usr/lib/x86_64-linux-gnu/libudev.so.0 --privileged=true myorg/clairscan ./clair-scanner --clair="http://$myip:6060" --ip=$myip $animage

However, I keep getting the below:

2018/01/24 09:50:57 [INFO] ▶ Start clair-scanner
2018/01/24 09:51:27 [INFO] ▶ Server listening on port 9279
2018/01/24 09:51:27 [INFO] ▶ Analyzing efd75b67cb976df08013762c6dc86092f41c07ca62f90d41291185703336d55d
2018/01/24 09:51:27 [CRIT] ▶ Could not analyze layer: Clair responded with a failure: Got response 400 with message {"Error":{"Message":"could not find layer"}}

Any suggestions?

Whitelist do not allow for expiry

My security team often asks that any exception has a built-in expiry date. A great example would be for a vulnerability that is not patched yet. So we can expect/whitelist the issue/vulnerability for 1 month, but we don't what to whitelist forever, or until manual review.

An example of what an expire config might look like.

generalwhitelist:
  CVE-2017-6055: XML
    expire: 2019-02-03
  CVE-2017-5586: OpenText
images:
  ubuntu:
    CVE-2017-5230: Java
    CVE-2017-5230: XSX
  alpine:
    CVE-2017-3261: SE

Analysis fails with 404 Not Found

I'm running Clair in a local docker-compose setup. clair-scanner fails with:

$ ./clair-scanner <my-repo>:5000/vsys/ies-base-armhf:2.0.2
2018/10/25 15:02:25 [INFO] ▶ Start clair-scanner
2018/10/25 15:02:35 [INFO] ▶ Server listening on port 9279
2018/10/25 15:02:35 [INFO] ▶ Analyzing 6740d70d00d7e3430d8b84463c3c2ab5f1b2e83b8340eccdc958deb209e0b317
2018/10/25 15:02:35 [CRIT] ▶ Could not analyze layer: Clair responded with a failure: Got response 404 with message Not Found

Could this possibly be because the repo requires authentication? But then why isn't the response 401 Unauthorized?

clair scanner not reading vulnerabilities from postgres databas

Hello,

I am using clair-scanner as below.


docker run --net=host -d --name db arminc/clair-db:2018-10-17
docker run --net=host --add-host postgres:127.0.0.1 -d --name clair --net=host arminc/clair-local-scan:v2.0.5

After this, i can see the data in the database.

root@G5:~/test7# docker exec -it db psql -p 5432 -h 127.0.0.1 -U postgres -W
Password for user postgres: 
psql (10.4)
Type "help" for help.

postgres=# select count(*) from vulnerability;
 count  
--------
 193799
(1 row)

postgres=#

But, when i run the below.
./clair-scanner --ip 127.0.0.1 -c "http://127.0.0.1:6060" node > out.out

I can see below.

2018/10/18 14:35:16 [INFO] ▶ Start clair-scanner
2018/10/18 14:35:29 [INFO] ▶ Server listening on port 9279
2018/10/18 14:35:29 [INFO] ▶ Analyzing 98f5a0e202a009cfef4103881ed9e5cbc32baf28829b19442ae736b1ac885dd4
2018/10/18 14:35:29 [INFO] ▶ Analyzing f4f196e4a99e1948178173e35b7e6a578aafd85b2f492ea5d50c64249627c5d6
2018/10/18 14:35:29 [INFO] ▶ Analyzing a367adb66eb77538c5be8545dc70a07ce9e2100b29fd26492e8db7c9751d48aa
2018/10/18 14:35:29 [INFO] ▶ Analyzing 8da6f90590bc48fe8c6cc31eaaf94783e4ff281de5a04e8f917e39dedb384701
2018/10/18 14:35:29 [INFO] ▶ Analyzing 322329db1f30f3f24cf4d81c92d9d70216690155da59340f4733ac48fd1b9fa7
2018/10/18 14:35:29 [INFO] ▶ Analyzing 16a3344b19540560ad6c24ce0fbd027dbe0640ef69e20e8f617f5095b1481337
2018/10/18 14:35:29 [INFO] ▶ Analyzing a6ca2fd079b44b4e39e4afe341ff6276a439aadf946d59ad2637a8b2860b8d33
2018/10/18 14:35:29 [INFO] ▶ Analyzing 27bb12affa29d598087edff47abf559c35245135c2277a6b1b7b1f3202fd8b2c
2018/10/18 14:35:29 [WARN] ▶ Image [node] contains 708 total vulnerabilities
2018/10/18 14:35:29 [ERRO] ▶ Image [node] contains 708 unapproved vulnerabilities

Shouldn't clair-scanner read from the database, and report almost 0 errors, or am i doing something wrong?
Thanks in advance.

Docker containers vulnerability scan with Clair

Hi,
based on http://blog.xebia.com/docker-containers-vulnerability-scan-clair/
I did:

docker run -p 5432:5432 -d --name db arminc/clair-db:2017-05-05

docker run -p 6060:6060 --link db:postgres -d --name clair arminc/clair-local-scan:v2.0.0-rc.0

But when I go for:

clair-scanner nginx:1.11.6-alpine example-nginx.yaml

my machine does not know clair-scanner, I never installed such a thing !
clair-scanner: command not found

what is the issue?

May I ask you to give me a guidance and configuration to run the database properly and run and configure the Clair correctly and at the end a way to push the image and getting the feedback from Clair on my local machine.

Is clair-scanner vulnerable to zip-slip?

In reviewing projects used locally such as clair-scanner, I took a brief look for any usage of archive/tar and looked to see if there was any extraction that failed to sanitize the path name being extracted and spotted:

clair-scanner/utils.go

Line 66 in ed09c54

path := filepath.Join(target, header.Name)

Essentially looking at the entire block:

	for {
		header, err := tarReader.Next()
		if err == io.EOF {
			break
		} else if err != nil {
			return err
		}

		path := filepath.Join(target, header.Name)
		info := header.FileInfo()
		if info.IsDir() {
			if err = os.MkdirAll(path, info.Mode()); err != nil {
				return err
			}
			continue
		}

		file, err := os.OpenFile(path, os.O_CREATE|os.O_TRUNC|os.O_WRONLY, info.Mode())
		if err != nil {
			return err
		}
		defer file.Close()
		if _, err = io.Copy(file, tarReader); err != nil {
			return err
}

and taking into account https://snyk.io/research/zip-slip-vulnerability#go I'm not seeing anything sanitize the path to make sure that header.Name didn't contain something like ../../../../../../../../etc/passwd to try and persuade the tool to extract to /etc/passwd irrespective of the provided target path.

Presumably one could mitigate this by ensuring you run via a container that is set 'read-only' and the scan extraction is under path provided by an attached volume and this should result in an error in attempting to create a file under path within the container.

Question: arminc/clair-db and arminc/clair-local-scan dockerfiles

Hello,

Where can we find the Dockerfiles of the images used in the Doc arminc/clair-db and arminc/clair-local-scan ?
I checked here: https://hub.docker.com/r/arminc/but find only the builds.

Thanks,

client version problem

When I run clair-scanner -w example-alpine.yaml --ip 127.0.0.1 alpine:3.5, it gives me th following error:
2018/02/16 13:07:34 [INFO] ▶ Start clair-scanner
2018/02/16 13:07:34 [CRIT] ▶ Could not save Docker image [alpine:3.5]: Error response from daemon: client version 1.36 is too new. Maximum supported API version is 1.35

Before that command I run the following two commands:
docker run -p 5431:5432 -d --name db arminc/clair-db:2018-02-15
docker run -p 6060:6060 --link db:postgres -d --name clair arminc/clair-local-scan:v2.0.1

What's the solution?

Whitelist ignored

Hello, it appears that whitelist is ignored.

my whitelist.yaml:

generalwhitelist: #Approve CVE for any image
  CVE-2017-12424: shadow

./clair-scanner_linux_amd64 -w scanner_config/whitelist.yaml --ip myip -c http://localhost:6060 myimage

the result still has listed: CVE-2017-12424

Using latest scanner v5

Change the exit code based on the presence of vulnerabilities

I would be helpful for integrations with CI's like jenkins if the tool exited with a non-zero exit code based on if there was a vulnerability in docker image scanned

clair-scanner command not found

docker run -p 5432:5432 -d --name db arminc/clair-db:2017-09-18

docker run -p 6060:6060 --link db:postgres -d --name clair arminc/clair-local-scan:v2.0.1
I ran these two commands successfully but I get a clair-scanner not found error when trying to execute the test image.
I am using docker for MAC.

Any help would be appreciated

Docker Image for clair-scanner ?

Any way to get an image for clair-scanner? So that it can be used it in a CI/CD situation without having to download it.

Exit code does not work

looks like related to #3, i'm unable to retrieve any other exit code than 0.

# ./clair-scanner_linux_amd64 --clair="http://clair.example.com:6060" --ip $(hostname -i) nginx
2017/10/18 13:04:36 [INFO] ▶ Start clair-scanner
2017/10/18 13:04:36 [INFO] ▶ Server listening on port 9279
2017/10/18 13:04:36 [INFO] ▶ Analyzing 036e48af29d6f44b505729e60a1ce93f51e5edb2db4adc93762539552cfd6d35
2017/10/18 13:04:36 [INFO] ▶ Analyzing faa943160189cd1cd48c9a154db90bb63ed669c6803416f04b582f12f36c84eb
2017/10/18 13:04:36 [INFO] ▶ Analyzing 01110f2de08518039bb67c0c64e18612bd54e54c46ced65fe6eb2fda39b60598
2017/10/18 13:04:36 [INFO] ▶ Analyzing 2ea5627793f03e363c7127b799febd133ba03ad564f7e7605084620a04d7094e
2017/10/18 13:04:36 [INFO] ▶ Unapproved vulnerabilities [[CVE-2017-0379]]
# echo $?
0

Sources of Vuln DB Repository/corpus

Hi , can you list me what are the sources from which the Vuln DB components are being updated in the Knowledgebase i.e. arminc/clair-db:XXXX-XX-XX ?

Create an Jenkins plugin for easy usage

Pre-build the docker build container for faster build time on travis

Clair responded with a failure: Got response 500 with message....

I am trying to scan docker images within a Jenkins Pipeline.

It starts to scan the containers - but during the scan I get the following error:

> 2018/05/10 19:03:37 [INFO] â–¶ Start clair-scanner
> 2018/05/10 19:03:45 [INFO] â–¶ Server listening on port 9279
> 2018/05/10 19:03:45 [INFO] â–¶ Analyzing 45d88eda03f89f18139930e9d3cbf869bf84f1fd776ed0a059788e2022339ef9
> 2018/05/10 19:03:45 [INFO] â–¶ Analyzing 646268fce3365f94e1e0598ab42b24c1c66ae961486f21db3d1a875a208e4561
> 2018/05/10 19:03:45 [INFO] â–¶ Analyzing d675e7b72207aba07b6ca08d5515108ae0110add18f7154f8eb5f39462d6f8f7
> 2018/05/10 19:03:45 [INFO] â–¶ Analyzing 10166da365caa812d5b86cf22660102945b7dd9160ada1f0c35720cc83cb8b0b
> 2018/05/10 19:03:45 [INFO] â–¶ Analyzing 7c786034d2aaf5a9c0a53f76245b12cdb6ac1dd0db496f9a16bc0e9bc0c47eb1
> 2018/05/10 19:03:45 [INFO] â–¶ Analyzing dc3a502d001b613ee58bf016e426fbcebc949603038d55ae67f16bcc34da93f4
> 2018/05/10 19:03:45 [INFO] â–¶ Analyzing 4741280194f1c829546eb866dcf93fb209028c6ea13004c419bed2ea6f877685
> 2018/05/10 19:03:45 [INFO] â–¶ Analyzing a4b2413ffe86b60f7bd58dcb541d0e724888aa50217d36205aaa6668f7d1cec2
> 2018/05/10 19:03:45 [INFO] â–¶ Analyzing 07c72f7d9c7f3758bf05793b46b86cf49a916eefde9d41862e910997804531bc
> 2018/05/10 19:03:45 [INFO] â–¶ Analyzing 2c9d10e4452339983909d51c69f6f2f87f87861b8f5bf61d219ea46faa9c1c0c
> 2018/05/10 19:03:45 [INFO] â–¶ Analyzing fc7175b0ec6feced5b08acd82022319bb5b4f64faa941cbeb7a4b7d71dd76cf2
> 2018/05/10 19:03:45 [INFO] â–¶ Analyzing 1100c2c3857fbd328ee4de17e8e7643bee63199275dd4cee891bfc38f85bd882
> 2018/05/10 19:03:45 [INFO] â–¶ Analyzing f49d1014c081c85b3f640c932e3d28ba85bd7903510b154ef641a5a237be0442
> 2018/05/10 19:03:45 [INFO] â–¶ Analyzing 1ddea952412225a1bfaf72daae8cacbfac9a85c2bce4690d10da8cc6dddc4eb6
> 2018/05/10 19:03:45 [INFO] â–¶ Analyzing b9aba1ab7aaedd493316f0fd0bb015a807ed0f1c1467a604e5c7976b3209e141
> 2018/05/10 19:03:45 [INFO] â–¶ Analyzing ef9a326f6b6e9e85a9cd47c0cb4cb4421c70c5345251efc09e7086a3543e0d6d
> 2018/05/10 19:03:45 [INFO] â–¶ Analyzing 0d00b1796e75233dafa0231bddcb42e7ae8d1b9942fcf15f2f1e5b70682191b6
> 2018/05/10 19:03:45 [INFO] â–¶ Analyzing b8fdf2f68d3b37e2e21e47d8a0b6a4711996b368d454d20eabf6c740d44c4bef
> 2018/05/10 19:03:45 [CRIT] â–¶ Could not analyze layer: Clair responded with a failure: Got response 500 with message {"Error":{"Message":"database: an error occured when querying the backend"}}

The logs from the clair-local-scan process are below:

{"Event":"running database migrations","Level":"info","Location":"pgsql.go:216","Time":"2018-05-10 09:13:55.300877"}
{"Event":"database migration ran successfully","Level":"info","Location":"pgsql.go:223","Time":"2018-05-10 09:13:55.311560"}
{"Event":"starting main API","Level":"info","Location":"api.go:52","Time":"2018-05-10 09:13:55.311758","port":6060}
{"Event":"starting health API","Level":"info","Location":"api.go:85","Time":"2018-05-10 09:13:55.311945","port":6061}
{"Event":"notifier service is disabled","Level":"info","Location":"notifier.go:77","Time":"2018-05-10 09:13:55.312095"}
{"Event":"updater service started","Level":"info","Location":"updater.go:81","Time":"2018-05-10 09:13:55.312090","lock identifier":"4ad113ae-6bce-4626-8c0f-87fb615c92fb"}
{"Event":"updating vulnerabilities","Level":"info","Location":"updater.go:182","Time":"2018-05-10 09:13:55.316496"}
{"Event":"fetching vulnerability updates","Level":"info","Location":"updater.go:228","Time":"2018-05-10 09:13:55.316580"}
{"Event":"Start fetching vulnerabilities","Level":"info","Location":"oracle.go:119","Time":"2018-05-10 09:13:55.316685","package":"Oracle Linux"}
{"Event":"Start fetching vulnerabilities","Level":"info","Location":"alpine.go:52","Time":"2018-05-10 09:13:55.316716","package":"Alpine"}
{"Event":"Start fetching vulnerabilities","Level":"info","Location":"debian.go:63","Time":"2018-05-10 09:13:55.316788","package":"Debian"}
{"Event":"Start fetching vulnerabilities","Level":"info","Location":"rhel.go:92","Time":"2018-05-10 09:13:55.316779","package":"RHEL"}
{"Event":"Start fetching vulnerabilities","Level":"info","Location":"ubuntu.go:88","Time":"2018-05-10 09:13:55.316709","package":"Ubuntu"}
{"Event":"finished fetching","Level":"info","Location":"updater.go:242","Time":"2018-05-10 09:13:56.405694","updater name":"alpine"}
{"Event":"finished fetching","Level":"info","Location":"updater.go:242","Time":"2018-05-10 09:13:57.730565","updater name":"rhel"}
{"Event":"finished fetching","Level":"info","Location":"updater.go:242","Time":"2018-05-10 09:13:59.169152","updater name":"debian"}
{"Event":"Handled HTTP request","Level":"info","Location":"router.go:57","Time":"2018-05-10 09:14:26.937475","elapsed time":3591444,"method":"POST","remote addr":"127.0.0.1:50818","request uri":"/v1/layers","status":"201"}
{"Event":"Handled HTTP request","Level":"info","Location":"router.go:57","Time":"2018-05-10 09:14:26.958169","elapsed time":19395274,"method":"POST","remote addr":"127.0.0.1:50820","request uri":"/v1/layers","status":"201"}
{"Description":"insertLayer","Event":"Handled Database Error","Level":"error","Location":"pgsql.go:284","Time":"2018-05-10 09:14:26.970918","error":"pq: insert or update on table \"layer\" violates foreign key constraint \"layer_parent_id_fkey\""}
{"Event":"Handled HTTP request","Level":"info","Location":"router.go:57","Time":"2018-05-10 09:14:26.971115","elapsed time":12498044,"method":"POST","remote addr":"127.0.0.1:50824","request uri":"/v1/layers","status":"500"}
> end log of container 'clair-local-scan' in pod 'jenkins-slave-8nhlw-rpl7s'

We have also looked at the clair-db to see what the DB is doing and we get the following error.

2018-05-10 12:40:14.627 UTC [10243] ERROR:  insert or update on table "layer" violates foreign key constraint "layer_parent_id_fkey"
2018-05-10 12:40:14.627 UTC [10243] DETAIL:  Key (parent_id)=(6) is not present in table "layer".
2018-05-10 12:40:14.627 UTC [10243] STATEMENT:  
                                             INSERT INTO Layer(name, engineversion, parent_id, namespace_id, created_at)
                   VALUES($1, $2, $3, $4, CURRENT_TIMESTAMP)
                   RETURNING id

Any ideas what the issue could be?

Rescan of image Error

If you can help me determine if this bug is related to something other than the clair-scanner than I would be very grateful to close this issue. But here is the issue. I updated my image (joeyn414/centosgauntlt) so I of course wanted to see an updated list of vulnerabilities against the image. I deleted the image from my machine and forced a new pull, after I did that I ran the clair-scanner again and I got this error:

panic: runtime error: index out of range

goroutine 15 [running]:
main.getVulnerabilities(0x7fff8c5997dc, 0x15, 0x0, 0x0, 0x0, 0x7fff8c5997dc, 0x15, 0x7fff8c5997f2, 0x9, 0x0)
	/root/go/src/vendor/clair-scanner/main.go:320 +0x60f
main.analyzeImage(0x7fff8c5997b1, 0x16, 0xc420262c80, 0x1c, 0x7fff8c5997dc, 0x15, 0x7fff8c5997f2, 0x9, 0x0, 0x0, ...)
	/root/go/src/vendor/clair-scanner/main.go:118 +0x36b
main.start.func1(0xc4201c3680, 0x7fff8c5997b1, 0x16, 0xc420262c80, 0x1c, 0x7fff8c5997dc, 0x15, 0x7fff8c5997f2, 0x9, 0x0, ...)
	/root/go/src/vendor/clair-scanner/main.go:82 +0xa7
created by main.start
	/root/go/src/vendor/clair-scanner/main.go:83 +0x1ea

Any idea what it could be? The error messages as you can see are not helpful.

Port already allocated error

I am successfully running clair on my docker host. When i run "docker run -p 6060:6060 --link db:postgres -d --name clair arminc/clair-local-scan:v2.0.1" , i get port already allocated error. 6060 clair container port is mapped to my host.
Does the clair-scanner work if i allocate a different port?

could not find layer

when trying to scan an image using one of the examples, the following occurs:

./clair-scanner_linux_amd64 nginx:1.11.6-alpine example-nginx.yaml http://127.0.0.1:6060 127.0.0.1
2017-09-19 11:11:22.019062 I | Analyzing 23f9039742dcfc7436c234a037aa1890b36b0a259e009687fe427aeb1691cd22
2017-09-19 11:11:23.022625 I | Analyzing faild: Could not analyze layer: Got response 400 with message {"Error":{"Message":"could not find layer"}}

Using clair-scanner behind corporate proxy

Hi!

I'm trying to scan my images behind corporate proxy, I've configured docker to use proxy and everything is working as expected but clair is getting timeouts when try to download databases.

See some logs output:

{"Event":"updating vulnerabilities","Level":"info","Location":"updater.go:182","Time":"2018-11-29 11:57:09.153969"}
{"Event":"fetching vulnerability updates","Level":"info","Location":"updater.go:228","Time":"2018-11-29 11:57:09.154718"}
{"Event":"Start fetching vulnerabilities","Level":"info","Location":"rhel.go:92","Time":"2018-11-29 11:57:09.176714","package":"RHEL"}
{"Event":"Start fetching vulnerabilities","Level":"info","Location":"oracle.go:119","Time":"2018-11-29 11:57:09.167287","package":"Oracle Linux"}
{"Event":"Start fetching vulnerabilities","Level":"info","Location":"alpine.go:52","Time":"2018-11-29 11:57:09.193891","package":"Alpine"}
{"Event":"Start fetching vulnerabilities","Level":"info","Location":"debian.go:63","Time":"2018-11-29 11:57:09.224470","package":"Debian"}
{"Event":"Start fetching vulnerabilities","Level":"info","Location":"ubuntu.go:85","Time":"2018-11-29 11:57:09.183037","package":"Ubuntu"}
{"Event":"could not download Oracle's update list","Level":"error","Location":"oracle.go:134","Time":"2018-11-29 11:57:39.180726","error":"Get https://linux.oracle.com/oval/: dial tcp 141.146.114.22:443: i/o timeout"}
{"Event":"an error occured when fetching update","Level":"error","Location":"updater.go:235","Time":"2018-11-29 11:57:39.180831","error":"could not download requested resource","updater name":"oracle"}
{"Event":"could not download RHEL's update list","Level":"error","Location":"rhel.go:106","Time":"2018-11-29 11:57:39.239028","error":"Get https://www.redhat.com/security/data/oval/: dial tcp 23.223.91.63:443: i/o timeout"}
{"Event":"an error occured when fetching update","Level":"error","Location":"updater.go:235","Time":"2018-11-29 11:57:39.239187","error":"could not download requested resource","updater name":"rhel"}
{"Event":"could not download Debian's update","Level":"error","Location":"debian.go:68","Time":"2018-11-29 11:57:39.264244","error":"Get https://security-tracker.debian.org/tracker/data/json: dial tcp 23.111.9.35:443: i/o timeout"}
{"Event":"an error occured when fetching update","Level":"error","Location":"updater.go:235","Time":"2018-11-29 11:57:39.264334","error":"could not download requested resource","updater name":"debian"}
{"Event":"could not download layer","Level":"warning","Location":"driver.go:130","Time":"2018-11-29 11:57:40.788924","error":"Get http://192.168.99.100:9279/8f52818719ad48a0af558ae2a44eed3cb3fe080f13c9fbdc67ef15667af59196/layer.tar: dial tcp 192.168.99.100:9279: connect: connection refused"}
{"Event":"failed to extract data from path","Level":"error","Location":"worker.go:122","Time":"2018-11-29 11:57:40.789310","error":"could not find layer","layer":"8f52818719ad48a0af558ae2a44eed3cb3fe080f13c9fbdc67ef15667af59196","path":"http://192.168.99.100:9279/8f52818719ad48a0af558ae2a44eed3cb3fe080f13c9fbdc67ef15667af59196/layer.tar"}

I'm running containers using host net:

docker run --net=host -d --name db arminc/clair-db:2018-11-28
docker run --net=host --add-host postgres:127.0.0.1 -d --name clair --net=host arminc/clair-local-scan:v2.0.6

Command used to scan image:

./clair-scanner --ip 192.168.99.100 -c "http://192.168.99.100:6060" arminc/clair-db:2018-11-28

Environment:
W10 pro
Docker toolbox -> Boot2Docker version 18.06.0-ce, build HEAD : 1f40eb2

change port for arminc/clair-db as a docker postgresql container is running on the same port

Hi, could you please let me know while running the command
docker run -p 5432:5432 -d --name db arminc/clair-db:2017-09-18 , i already have a postgres container for my project running on that 5432 port, if i simple change
docker run -p 5432:5432 -d --name db arminc/clair-db:2017-09-18 this port number to
docker run -p XXXX:XXXX -d --name db arminc/clair-db:2017-09-18 would the scanner work fine or does it gives any error?

Get output from clair-scanner as a json file

Can you make sure that your vulnerability report can be provided as a json file?

Make the http server port configurable

The port is now fixed make it configurable.
How to handle when the port can't be used?

Incorrect usage error thrown when trying to setup a dedicated clair server

I can't seem to be able to get clair-scanner to work with a dedicated clair server in a separate pod within the same kubernetes cluster. I've tried -c / --clair . None of them work. Using --ip can get it to at least acknowledged the server but its print out is referencing 127.0.0.1 localloop back address. Does clair-scanner only work with localhost?

Also, clair-scanner should have a debug logging verbose option -v which it hasn't had yet.

See steps below.

root@gitlab-runner-2668289193-7skhk:/# ./clair-scanner -v -c "http://clair:6060" 76d05ed08bb4
Error: incorrect usage

Usage: clair-scanner [OPTIONS] IMAGE

Scan local Docker images for vulnerabilities with Clair

Arguments:
  IMAGE=""     Name of the Docker image to scan

Options:
  -w, --whitelist=""                    Path to the whitelist file
  -t, --threshold="Unknown"             CVE severity threshold. Valid values; 'Defcon1', 'Critical', 'High', 'Medium', 'Low', 'Negligible', 'Unknown'
  -c, --clair="http://127.0.0.1:6060"   Clair URL
  --ip="localhost"                      IP address where clair-scanner is running on
  -l, --log=""                          Log to a file
  --all, --reportAll=true               Display all vulnerabilities, even if they are approved
  -r, --report=""                       Report output file, as JSON
root@gitlab-runner-2668289193-7skhk:/# ping clair
PING clair.default.svc.cluster.local (10.3.244.231) 56(84) bytes of data.
^C
--- clair.default.svc.cluster.local ping statistics ---
2 packets transmitted, 0 received, 100% packet loss, time 999ms

root@gitlab-runner-2668289193-7skhk:/# ./clair-scanner --ip="10.3.244.231" 76d05ed08bb4
2018/06/08 16:17:45 [INFO] ▶ Start clair-scanner
2018/06/08 16:17:52 [INFO] ▶ Server listening on port 9279
2018/06/08 16:17:52 [INFO] ▶ Analyzing c212ed5a00ac395a0b508281c9692bee405948ace8c85c0f8f3feef71f9f7418
2018/06/08 16:17:52 [CRIT] ▶ Could not analyze layer: POST to Clair failed Post http://127.0.0.1:6060/v1/layers: dial tcp 127.0.0.1:6060: getsockopt: connection refused

Scan fails: server has client authentication enabled

hi
when running the clair-scanner I am encountering some issue in how our docker service is setup. Simply put, it is configured with client certificate authentication and when trying to scan a imge I'm getting:

user: ./clair-scanner_linux_386 --clair="http://10.5.74.146:6060" --ip="10.5.74.146" clair
2018/11/15 15:15:59 [INFO] ▶ Start clair-scanner
2018/11/15 15:15:59 [CRIT] ▶ Could not save Docker image [clair]: The server probably has client authentication (--tlsverify) enabled. Please check your TLS client certification settings: Get https://127.0.0.1:2376/v1.25/images/get?names=clair: remote error: tls: bad certificate

I tried to change the clair-local-scanner to run with -insecure-tls but this did not slve anything.

Anyone would know how to address this?

Image scoring

Hello, I use clair-scanner to complete the local static scanning. I want to know how to score the image, such as attachment information.

python:2.7.txt

Allow header to be send to Clair

When using your own Clair the chance is that it is secured, allow a way to send a header when communicating to Clair that passes the authentication information

Make a version that works agains future clair v3

Possible to extend whitelist to support tags?

We currently use multiple tags of base images for different apps. For example, we might have some apps running debian:jessie and others running debian:stretch.

Would it be possible to extend the whitelist to be aware of tags? This would allow us to mark certain CVEs as ok if they were to appear in jessie (since Debian hasn't provided a fix), but not mark them as ok in stretch, where a fix has been released.

Make automatic releases with travis

[CRIT] ▶ Could not analyze layer: Clair responded with a failure: Got response 400 with message {"Error":{"Message":"could not find layer"}}

clair-scanner --ip=<host_ip_address> --threshold=Medium --report=filename.json nginx:latest
2018/12/07 01:38:13 [INFO] ▶ Start clair-scanner
2018/12/07 01:38:17 [INFO] ▶ Server listening on port 9279
2018/12/07 01:38:17 [INFO] ▶ Analyzing 5890f2d38d34ce375d44a098dc120817201a6903df1b9dfc70b3ea86e4ffb175
2018/12/07 01:38:18 [CRIT] ▶ Could not analyze layer: Clair responded with a failure: Got response 400 with message {"Error":{"Message":"could not find layer"}}

docker ps | grep clair
43b784d3d7d6 arminc/clair-db:latest "docker-entrypoint.s…" 27 minutes ago Up 27 minutes 0.0.0.0:5432->5432/tcp db
e43ebdc7fc2c arminc/clair-local-scan:v2.0.1 "/clair -config=/con…" 2 hours ago Up 2 hours 0.0.0.0:6060->6060/tcp, 6061/tcp clair

docker logs e43ebdc7fc2c

{"Event":"could not download layer","Level":"warning","Location":"driver.go:129","Time":"2018-12-07 06:38:18.894778","error":"Get http://10.71.11.70:9279/5890f2d38d34ce375d44a098dc120817201a6903df1b9dfc70b3ea86e4ffb175/layer.tar: dial tcp 10.71.11.70:9279: getsockopt: no route to host"}
{"Event":"failed to extract data from path","Level":"error","Location":"worker.go:122","Time":"2018-12-07 06:38:18.895004","error":"could not find layer","layer":"5890f2d38d34ce375d44a098dc120817201a6903df1b9dfc70b3ea86e4ffb175","path":"http://10.71.11.70:9279/5890f2d38d34ce375d44a098dc120817201a6903df1b9dfc70b3ea86e4ffb175/layer.tar"}
{"Event":"Handled HTTP request","Level":"info","Location":"router.go:57","Time":"2018-12-07 06:38:18.896015","elapsed time":1013364278,"method":"POST","remote addr":"172.17.0.1:36072","request uri":"/v1/layers","status":"400"}

While running clair-scanner command, 9279 port is opened on the host machine and gets closed before download is started.

Can you please help on this.

Or maybe have a new threshold setting for failure level. This could leave lower severity items as Unapproved, but not return a failed exit code.