Like atomic scan, is it possible to write custom scanner that would work with image-inspector? I'd like to build images using OpenShift and scan them as well. The scan results need to be made available to the end-user - example (json files on the link are the results of various scanners.)

Also, pardon my ignorance but, how is image-inspector different from atomic scan? I ask this because it seems to me like both have built-in scanners based on OpenSCAP project.

README suggests to use --html option:

An HTML OpenSCAP scan report will be served on <serve_path>/api/v1/openscap-report if the --html
option is used.

But there seems to be no such option:

$ docker run -ti --rm --privileged -p 8080:8080 \
  -v /var/run/docker.sock:/var/run/docker.sock \
  openshift/image-inspector --image=fedora:20 --html \
  --path=/tmp/image-content --serve 0.0.0.0:8080
flag provided but not defined: -html
Usage of /usr/bin/image-inspector:
  -chroot
        Change root when serving the image with webdav
  -cve-url string
        An alternative URL source for CVE files (default "https://www.redhat.com/security/data/metrics/ds/")
  -docker string
        Daemon socket to connect to (default "unix:///var/run/docker.sock")
  -dockercfg value
        Location of the docker configuration files. May be specified more than once (default [])
  -image string
        Docker image to inspect
  -openscap-html-report
        Generate an OpenScap HTML report in addition to the ARF formatted report
  -password-file string
        Location of a file that contains the password for authentication with the docker registry
  -path string
        Destination path for the image files
  -scan-results-dir string
        The directory that will contain the results of the scan
  -scan-type string
        The type of the scan to be done on the inspected image. Available scan types are: [openscap]
  -serve string
        Host and port where to serve the image with webdav
  -username string
        username for authenticating with the docker registry

from https://github.com/openshift/image-inspector/pull/72/files#r137217513

We should include a gofmt check in CI/CD

When running image-inspector I sometimes get:

2016/06/16 11:28:46 Unable to scan image: Unable to run OpenSCAP: OpenSCAP error: 1: exit status 1
Input:
[xccdf eval --results-arf /var/tmp/image-inspector-scan-results-727398075/results-arf.xml /tmp/com.redhat.rhsa-RHEL6.ds.xml.bz2]
Output:
url:1: parser error : Document is empty
OpenSCAP Error: Could not read from bZ2FILE: DATA_ERROR_MAGIC [bz2.c:83]

In the log I am missing what url was used to download com.redhat.rhsa-RHEL6.ds.xml.bz2 and other useful information.

cc @enoodle

Currently the (anti-)pattern of

if err != nil {
     return nil, err
}

is replete through our repository. this pattern makes debugging errors very difficult when they occur, as we're not provided a stack trace or any context of where the original error occurred. for example, see https://bugzilla.redhat.com/show_bug.cgi?id=1486314 where the only error message we can see is error: unexpected EOF. Where did this EOF happen? Was it during a file download? Opening a file? Writing to disc? Extracting an archive?

I think it's an anti-pattern in go to bubble up errors without providing context for them. For this reason packages such as https://github.com/pkg/errors have been created so that lower-level errors can be wrapped before they are returned up the call stack.

I propose that we improve the debuggability of image-inspector by wrapping errors wherever they are returned, be it with pkg/error, fmt.Errorf("...: %v", err), or some other solution.

I try to create jenkins slave to run image-inspector to scan image but i cant do that.

Could you share me how to integrate with jenkins slave for pipeline?
if cant do that, i'll change my solution to scan docker image.

problem

• cant run docker image in jenkins slave.
• but if run image-inspector as openshift container , we can scan image

Thanks

When automatically retrieving the https://www.redhat.com/security/data/metrics/ds/com.redhat.rhsa-RHEL7.ds.xml.bz2 file to /tmp/ the contents are invalid and contain: Perhaps there are invalid headers being inserted in the HTTP get but I do not have a way to debug this because there are no verbose loggings.

<HTML><HEAD>
<TITLE>Access Denied</TITLE>
</HEAD><BODY>
<H1>Access Denied</H1>
You don't have permission to access "http&#58;&#47;&#47;www&#46;redhat&#46;com&#47;security&#47;data&#47;metrics&#47;ds&#47;com&#46;redhat&#46;rhsa&#45;RHEL7&#46;ds&#46;xm1&#46;b22" on this server.<P>
Reference&#32;&#35;18&#46;5e962317&#46;1 36093256&#46;1295892e
</BODY>
</HTML>

I am on a mac and running

$ make
hack/build-go.sh

Detected Go version: go version go1.11.2 darwin/amd64.
image-inspector builds require Go version 1.5 or greater.

I am new to go but 1.11 > 1.5 unless I am missing something.

Since we are going to add more scan types (container scan, clamav scan, etc) we need to agree on unified result format for the scans that can be consumed by third-party projects (like openshift).

In openshift we will most likely want to add annotations for every image we scan and present this information to users. The scan can indicate various markers about the image quality, like how many CVE the image/containers contains, what is its clamav score, etc...

The proposed OpenShift image annotation format seems to be:

quality.images.openshift.io/<quality type>.<provider/scan type>: {}
// example:

quality.images.openshift.io/vulnerability.openscap: {}

However, OpenShift have size limit of how many data can be stored in the annotations. In case of openscap, the result of the scan can be ~17MB which won't fit this size limit.

For that we should probably just extract the failures from the openscap report (which will involve parsing the ARF result file and extracting them...).

The proposed JSON result object:

{
  "startedAt": "00000",
  "finishedAt": "111111",
  "imageName": "registry.access.redhat.com/foo/bar:latest",
  "imageID": "sha256:xxxx",
  "results": [
    {
      "name": "openscap",
      "scannerVersion": "openscap 1.2.10",
      "timestamp": "11111",
      "reference": "https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2017-7502",
      "description": "RHSA-2017:1365: nss security and bug fix update (Important)",
      "summary": [ {"label": "important"} ],
    },
    {
      "name": "clamav",
      "scannerVersion": "clamav 1.2.3",
      "timestamp": "11111",
      "reference": "clamav.scanner.openshift.io/definitions/123",
      "description": "bitcoin miner",
      "summary": [ {"label": "important"} ],
    },
  ],
}

We would need the imageName and imageID to make sure the Docker image we are going to attach this annotation to match with the one we scanned.

We should provide release binaries people can just download and use. Using this via docker image is cool, but if somebody wants to try this out and he has to deal with godeps, he will probably freak out ;-)

@simon3z @pweil- FYI

There is an image-inspector package available in the openshift-rhel channels. Is there one in a fedora/centos channel?

Currently trying to replace atomic scan tool with something that can run better within openshift and this tool almost fits the bill completely.

Something that is missing is the ability to easily get at the data that is currently being pushed via the post-results-url flag. Could this data be exposed:

1. via an endpoint such as /results when application/json is set as the content-type requested if --serve is passed as a parameter
2. via writing to disk if a new flag is provided that would point to a directory in the filesystem

Use case flow:

1. Jenkins pipeline that builds an s2i image within a Jenkins slave running on openshift and pushes the image to some externally available registry
2. Jenkins pipeline then calls stage to scan the image via accessing the openshift's host docker socket and then saves json results to a directory where Jenkins can further process (iterate over results and identify vulnerabilities etc)

Maybe something along the lines of this mocked up command

docker run -ti --rm -p 8080:8080 \
  -v /var/run/docker.sock:/var/run/docker.sock \
  -v /home/my.username/config:/config \
  registry.access.redhat.com/openshift3/image-inspector:v3.8.36-5 --image=my-docker-image:tag \
  --scan-type=openscap \
  --username=my.username \
  --password-file=/config/psswd
  --json-results-dir=/results

When the image is not present on the node the pull is failing:

# docker run --rm -ti --privileged -p 8080:8080 -v /var/run/docker.sock:/var/run/docker.sock image-inspector:2.0.0-1 --chroot --image=docker.io/fedora:23 --scan-type=openscap --serve=0.0.0.0:8080
2016/05/10 16:52:05 Pulling image docker.io/fedora:23
2016/05/10 16:52:05 Error inspecting image: Unable to create docker container: no such image

cc @enoodle @moolitayer

This is a critical and urgent issue.

Suggestion: since we'll now also always have a default empty auth we could warn but not fail if errors are encountered when loading auth files. If the image still fails to pull we'll eventually have a hard error.

https://github.com/openshift/image-inspector/blob/master/pkg/inspector/image-inspector.go#L358

@simon3z @enoodle WDYT?

This issue is to track indexing image-inspector repository to build at CentOS community container pipeline service.

A successful build indexes image at https://registry.centos.org/

Docker pull URL like registry.centos.org/namespace/image-inspector

Since centos:7 is used as a base image, the Dockerfile should not need many changes to onboard project on build service.

With project configured to build at pipeline service, there will be automatic rebuilds of image, when

• there is a change pushed to the repository
• the underlying base image got an update

To index this project to build at pipeline service, all it needs is to add a YAML file cccp.yml in the project repository with

job-id: image-inspector

Recent reports from image-inspector oscap provide a minified HTML document with css-minify and jquery bundled. This causes syntax parsing issues when go-xmldom is used and fails the parsing stage.

Requiring access to the docker.socket is a lot of privilege. With the https://github.com/containers/image/ library this should not be required.

Need to move this to a defer

https://github.com/openshift/image-inspector/blob/master/pkg/inspector/image-inspector.go#L336-L344

Running:

# cadaver http://127.0.0.1:8080/api/v1/content/

And listing etc I get:

dav:/api/v1/content/> ls etc
Listing collection `/api/v1/content/etc/': failed:
XML parse error at line 1: junk after document element

and on the server side:

2016/05/16 08:23:53 http: multiple response.WriteHeader calls

cc @enoodle

It was mentioned to me that a user can use the parameter to give an unsecured "http://" URL.
I suggest that this parameter will be changed to expect a URL without a protocol and use it as https.

I don't know exactly what versions of docker are affected but with docker-1.12.3-9.git47e22f2.fc25.x86_64 the extraction is not working, in fact image-inspector blocks on:

$ ./image-inspector --image=docker.io/nginx:latest --serve=0.0.0.0:8080
2016/11/29 13:13:59 Pulling image docker.io/nginx:latest
2016/11/29 13:14:02 Finished Downloading Image (0Kb downloaded)
2016/11/29 13:14:02 Extracting image docker.io/nginx:latest to /var/tmp/image-inspector-676612778

and never finishes (nothing is extracted in /var/tmp/image-inspector-676612778).

I think it's because we have to replace the obsolete CopyFromContainer with DownloadFromContainer.

Once we'll switch we'll have to make sure that the new API call works for Fedora, CentOS and RHEL.

cc @enoodle @pweil-

Hi

I am trying to run the image-inspector - version 2.1 on an OpenShift Origin installation where the image-inspector is launched through ManageIQ as pods on the namespace where I also have ManageIQ running.

So I would be running this image of dockerhub: openshift/image-inspector:2.1

The image-inspector starts up fine, can download and extract the image I request to be scanned with "Smartstate Analysis" in ManageIQ, but the image-inspector is complaining that it cannot determine the RHEL distribution that I am running on - as this log extract is showing.

...
OpenSCAP scanning /var/tmp/image-inspector-079165993. Placing results in /var/tmp/image-inspector-scan-results-896793972
2017/03/14 09:02:47 Unable to scan image: Unable to run OpenSCAP: Unable to get RHEL distribution number: could not find RHEL dist
...

Any good suggestions? Would the serviceaccount on OpenShift that is used to launch the image-inspector docker container need some more privileges, or is there maybe something else wrong?

I am running the OpenShift Origin instance on RHEL 7.3.

Best regards
Lars Milland

For example. running something like:
image-inspector -scan-type=clamav -clam-socket=/host/run/clamd.scan/clamd.sock -container=5ca8650f8245 -alsologtostderr -log_dir=/var/log/clam/

only returns this. with none of the actual scan results.:
2017/12/08 23:47:55 clamav scan took 117s (1 problems found)

I tried several variations of this, with a combination of every parameter and verbosity level I could find supported by image-inspector, and it seems only -post-results-url works as expected for me. I have been sending the results with -post-results-url successfully to a different tool that the catches the results, but it image-inspector doesn't log directly to disk or stderr when including that parameter, either.

Can anyone confirm logfile / stderror logging works when scanning running containers? Or provide an example in the event this could still be user error?

Thanks!

It should be possible to run OpenSCAP scans even in network-isolated clusters. Maybe by defining an internal URL from which to take the CVE definitions.
(This may be dependent on our ability that the CVE definitions can be trusted, there should be a BZ somewhere, @mpreisler should be able help us here).

@enoodle @pweil- it may be worth start brainstorming on this.
cc @smarterclayton @deads2k

When I run make after a fresh clone of the project I receive this error:

┌[dali]  image-inspector/ (master $=)
└> make
hack/build-go.sh
cmd/image-inspector.go:10:2: cannot find package "github.com/openshift/image-inspector/pkg/api" in any of:
        /usr/lib/golang/src/github.com/openshift/image-inspector/pkg/api (from $GOROOT)
        /src/github.com/openshift/image-inspector/pkg/api (from $GOPATH)
cmd/image-inspector.go:11:2: cannot find package "github.com/openshift/image-inspector/pkg/cmd" in any of:
        /usr/lib/golang/src/github.com/openshift/image-inspector/pkg/cmd (from $GOROOT)
        /src/github.com/openshift/image-inspector/pkg/cmd (from $GOPATH)
cmd/image-inspector.go:12:2: cannot find package "github.com/openshift/image-inspector/pkg/inspector" in any of:
        /usr/lib/golang/src/github.com/openshift/image-inspector/pkg/inspector (from $GOROOT)
        /src/github.com/openshift/image-inspector/pkg/inspector (from $GOPATH)
!!! Error in hack/../hack/common.sh:83
        'go install "cmd/image-inspector.go"' exited with status 1
Call stack:
        1: hack/../hack/common.sh:83 ii::build::build_binaries(...)
        2: hack/build-go.sh:15 main(...)
Exiting with status 1
!!! Error in hack/../hack/common.sh:72
        '( ii::build::setup_env; local platform="local"; export GOBIN="${II_OUTPUT_BINPATH}/${platform}"; mkdir -p "${II_OUTPUT_BINPATH}/${platform}"; go install "cmd/image-inspector.go" )' exited with status 1
Call stack:
        1: hack/../hack/common.sh:72 ii::build::build_binaries(...)
        2: hack/build-go.sh:15 main(...)
Exiting with status 1
make: *** [Makefile:17: all] Error 1

If I comment out the resetting of $GOHOME in hack/common.sh:67 and symlink the working dir into my own $GOPATH the build works.

I feel like I'm missing a step beyond just clone then make.

When an image is pulled:

2016/05/16 08:34:45 Pulling image docker.io/fedora:latest
...

there's no progress report.

We should log progress at constant intervals.

I haven't double checked the current behavior but I have the feeling that at the moment image-inspector is not trying to pull the latest image from the registry.

We should add an option (and discuss what's the default beahvir) to try and pull the latest image from the registry.

cc @enoodle @pweil-