Code Monkey home page Code Monkey logo

nvdtools's Introduction

Tests

NVD Tools

A collection of tools for working with National Vulnerability Database feeds.

The HOWTO provides a broader view on how to effectively use these tools.



Requirements

  • Go 1.13 or newer

Installation

You need a properly setup Go environment.

Download and install NVD Tools:

For Go 1.13 - 1.14:

go get github.com/facebookincubator/nvdtools/...
cd "$GOPATH"/src/github.com/facebookincubator/nvdtools/cmd
go install ./...

From Go 1.15 onwards, modules are not downloaded to GOPATH, but to GOMODCACHE. It is recommended to clone the repo and run run go install from there instead:

git clone https://github.com/facebookincubator/nvdtools
cd nvdtools
go install ./...

From Go 1.17 onwards, go get is deprecated. go install is used instead to download the module to the cache and install it:

go install github.com/facebookincubator/nvdtools/...@latest

How-build

go mod init github.com/facebookincubator/nvdtools
go mod tidy
make
cp build/bin/* ~/go/bin/

Command line tools

cpe2cve

cpe2cve is a command line tool for scanning an inventory of CPE names for vulnerabilities.

It expects a stream of lines of delimiter-separated fields, one of these fields being a delimiter-separated list of CPE names in the inventory.

Vulnerability feeds should be provided as arguments to the program in JSON format.

Output is a stream of delimiter-separated input value decorated with a vulnerability ID (CVE) and a delimiter-separated list of CPE names that match this vulnerability.

Unwanted input fields could be erased from the output with -e option.

Input and output delimiters can be configured with -d, -d2, -o an -o2 options.

The column to which output the CVE and matches for that CVE can be configured with -cve and -matches options correspondingly.

download data

curl -o- -s -k -v https://nvd.nist.gov/vuln/data-feeds >data-feeds.html
cat data-feeds.html|grep  -Eo '(/feeds\/[^"]*\.gz)'|xargs -I % wget -c https://nvd.nist.gov%

Example 1: scan a software for vulnerabilities

echo "cpe:/a:apache"|cpe2cve -cpe 1 -e 1 -cve 1  nvdcve-1.1-*.json.gz
echo "cpe:/a:gnu:glibc:2.28" | cpe2cve -cpe 1 -e 1 -cve 1 nvdcve-1.0-*.json.gz
CVE-2009-4881
CVE-2015-8985
CVE-2016-4429
CVE-2010-3192
CVE-2010-4756

Example 2: find vulnerabilities in software inventory per production host

./cpe2cve -d ' ' -d2 , -o ' ' -o2 , -cpe 2 -e 2 -matches 3 -cve 2 nvdcve-1.0-*.json.gz << EOF
host1.foo.bar cpe:/a:gnu:glibc:2.28,cpe:/a:gnu:zlib:1.2.8
host2.foo.bar cpe:/a:gnu:glibc:2.28,cpe:/a:haxx:curl:7.55.0
EOF
host1.foo.bar CVE-2009-4881 cpe:/a:gnu:glibc:2.28
host1.foo.bar CVE-2016-4429 cpe:/a:gnu:glibc:2.28
host2.foo.bar CVE-2014-5119 cpe:/a:gnu:glibc:2.28
host2.foo.bar CVE-2016-4429 cpe:/a:gnu:glibc:2.28
host2.foo.bar CVE-2018-1000120 cpe:/a:haxx:curl:7.55.0
host2.foo.bar CVE-2018-1000122 cpe:/a:haxx:curl:7.55.0
host2.foo.bar CVE-2010-4756 cpe:/a:gnu:glibc:2.28
host2.foo.bar CVE-2017-8817 cpe:/a:haxx:curl:7.55.0

csv2cpe

csv2cpe is a tool that generates an URI-bound CPE from CSV input, flags configure the meaning of each input field:

  • -cpe_part -- identifies the class of a product: h for hardware, a for application and o for OS
  • -cpe_vendor -- identifies the person or organisation that manufactured or created the product
  • -cpe_product -- describes or identifies the most common and recognisable title or name of the product
  • -cpe_version -- vendor-specific alphanumeric strings characterising the particular release version of the product
  • -cpe_update -- vendor-specific alphanumeric strings characterising the particular update, service pack, or point release of the product
  • -cpe_edition -- capture edition-related terms applied by the vendor to the product; this attribute is considered deprecated in CPE specification version 2.3 and it should be assigned the logical value ANY except where required for backward compatibility with version 2.2 of the CPE specification.
  • -cpe_swedition -- characterises how the product is tailored to a particular market or class of end users
  • -cpe_targetsw -- characterises the software computing environment within which the product operates
  • -cpe_targethw -- characterises the software computing environment within which the product operates
  • -cpe_language -- defines the language supported in the user interface of the product being described; must be valid language tags as defined by [RFC5646]
  • -cpe_other -- any other general descriptive or identifying information which is vendor- or product-specific and which does not logically fit in any other attribute value

Omitted parts of the CPE name defaults to logical value ANY, as per specification

Optional flag -lower brings the strings to lower case.

Example: generate URI-bound CPE name out of comma-separated list of attributes

$ echo 'a,Microsoft,Internet Explorer,8.1,SP1,-,*' | csv2cpe -x -lower -cpe_part=1 -cpe_vendor=2 -cpe_product=3 -cpe_version=4 -cpe_update=5 -cpe_edition=6 -cpe_language=7
cpe:/a:microsoft:internet_explorer:8.1:sp1:-

fireeye2nvd

fireeye2nvd downloads the vulnerability data from FireEye and converts it into NVD format. The resulting file can be used as a feed in cpe2cve processor

flexera2nvd

flexera2nvd downloads the vulnerability data from Flexera and converts it into NVD format. The resulting file can be used as a feed in cpe2cve processor

idefense2nvd

idefense2nvd downloads the vulnerability data from Idefense and converts it into NVD format. The resulting file can be used as a feed in cpe2cve processor

nvdsync

nvdsync synchronizes NVD data feeds to local directory; it checks the hashes of the files against the ones provided by NVD and only updates the changed files.

rpm2cpe

rpm2cpe takes a delimiter-separated input with one of the fields containing RPM package name and produces delimiter-separated output consisting of the same fields plus CPE name parsed from RPM package name.

Example: generate URI-bound CPE name out of RPM package filename

echo openoffice-eu-writer-4.1.5-9789.i586.rpm | rpm2cpe -rpm=1 -cpe=2 -e=1
cpe:/a::openoffice-eu-writer:4.1.5:9789:~~~~i586~

rustsec2nvd

rustsec2nvd converts the vulnerabilities from the Rustsec Advisory-DB into NVD format. The resulting file can be used as a feed in cpe2cve processor

snyk2nvd

snyk2nvd downloads the vulnerability data from Snyk and converts it into NVD format. The resulting file can be used as a feed in cpe2cve processor

vfeed2nvd

vfeed2nvd converts the vulnerability data from vFeed into NVD format. The resulting file can be used as a feed in cpe2cve processor

vulndb

vulndb is a command line tool to manage NVD-like vulnerability databases, backed by MySQL.

Supports NVD CVE JSON 1.0 feeds. Data is versioned, organized by provider names and grouped by vendor, custom, and snoozes datasets:

  • Vendor dataset: read-only CVE feeds we continuously import.
  • Custom dataset: allows to overwrite CVEs from vendor data with custom data during exports
  • Snooze dataset: user-defined CVE and metadata with deadline, used for remediation automation

See vulndb help for details.

Libraries

cvss2

Implementation of CVSS v2 specification which provides functions for serializing and deserializing vectors as well as score calculation.

cvss3

Implementation of CVSS v3 specification which provides functions for serializing and deserializing vectors as well as score calculation.

License

nvdtools licensed under Apache License, Version 2.0, as found in the LICENSE file.

nvdtools's People

Contributors

akint avatar alelevinas avatar amadman avatar andreacampi avatar andrestc avatar anr avatar crazyleg avatar daggia avatar dlespiau avatar epicfaace avatar fiorix avatar hktalent avatar hrenicfb avatar imanolbarba avatar jokliu avatar mrinaldhar avatar n1chre avatar opsengine avatar paralax avatar rtann avatar sadhvi25 avatar schischi avatar scottcunningham avatar skogtwin avatar testwill avatar thomas-maurice avatar viktorbarzin avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

nvdtools's Issues

Released tarball has non-standard binary names

I downloaded the MacOS release and found this inside:

Downloads $ tar zvft nvdtools-v0.1.0-darwin-amd64.tar.gz
-rwxrwxr-x travis/travis 3457960 2019-01-23 19:02 binaries/cpe2cve-v0.1.0-darwin-amd64
-rwxrwxr-x travis/travis 2327416 2019-01-23 19:02 binaries/csv2cpe-v0.1.0-darwin-amd64
-rwxrwxr-x travis/travis 8587736 2019-01-23 19:02 binaries/nvdsync-v0.1.0-darwin-amd64
-rwxrwxr-x travis/travis 2323176 2019-01-23 19:02 binaries/rpm2cpe-v0.1.0-darwin-amd64

I believe that suffix starting after the first dash, with version, OS and arch, isn't needed.

CVE-2021-45967 should not be there

Hello!

Maybe I am wrong but I think that CVE-2021-45967 should not be found...

echo cpe:/a::audiofile:0.3.6 | cpe2cve -cpe 1 -cve 1 feeds/nvd/*.json.gz
CVE-2017-6837   cpe:/a::audiofile:0.3.6
CVE-2017-6830   cpe:/a::audiofile:0.3.6
CVE-2017-6836   cpe:/a::audiofile:0.3.6
CVE-2017-6831   cpe:/a::audiofile:0.3.6
CVE-2017-6839   cpe:/a::audiofile:0.3.6
CVE-2017-6828   cpe:/a::audiofile:0.3.6
CVE-2017-6832   cpe:/a::audiofile:0.3.6
CVE-2021-45967  cpe:/a::audiofile:0.3.6
CVE-2017-6834   cpe:/a::audiofile:0.3.6
CVE-2017-6827   cpe:/a::audiofile:0.3.6
CVE-2017-6833   cpe:/a::audiofile:0.3.6
CVE-2017-6838   cpe:/a::audiofile:0.3.6
CVE-2017-6829   cpe:/a::audiofile:0.3.6
CVE-2017-6835   cpe:/a::audiofile:0.3.6

If I specify vendor it is not:

echo cpe:/a:audiofile:audiofile:0.3.6 | cpe2cve -cpe 1 -cve 1 feeds/nvd/*.json.gz
CVE-2017-6832   cpe:/a:audiofile:audiofile:0.3.6
CVE-2017-6836   cpe:/a:audiofile:audiofile:0.3.6
CVE-2017-6833   cpe:/a:audiofile:audiofile:0.3.6
CVE-2017-6828   cpe:/a:audiofile:audiofile:0.3.6
CVE-2017-6829   cpe:/a:audiofile:audiofile:0.3.6
CVE-2017-6834   cpe:/a:audiofile:audiofile:0.3.6
CVE-2017-6827   cpe:/a:audiofile:audiofile:0.3.6
CVE-2017-6838   cpe:/a:audiofile:audiofile:0.3.6
CVE-2017-6830   cpe:/a:audiofile:audiofile:0.3.6
CVE-2017-6839   cpe:/a:audiofile:audiofile:0.3.6
CVE-2017-6837   cpe:/a:audiofile:audiofile:0.3.6
CVE-2017-6831   cpe:/a:audiofile:audiofile:0.3.6
CVE-2017-6835   cpe:/a:audiofile:audiofile:0.3.6

Should a vendor be always specified?

Explain what -idxd does

Hello,
I don't understand what -idxd param in cpe2cve does.
The usage output says "build and use an index for CVE dictionary: increases the processing speed, but might miss some matches". But is it possible to know more about these miss?
Would it be possible to add some explanation somewhere in the doc?
Thanks!

Incorrect CVSS v3 base score equation

I noticed a problem with the following vector: CVSS:3.1/AV:L/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:N

According to https://nvd.nist.gov/vuln-metrics/cvss/v3-calculator and https://www.first.org/cvss/calculator/3.1, the base score should be 0.0; however, the nvdtools library gives 2.6. That is because of https://github.com/facebookincubator/nvdtools/blob/main/cvss3/score.go#L61: this should be a i <= 0 instead of just i < 0.

For this vector, we correctly get v.impactScore() = 0.0 and v.exploitabilityScore() = 2.515145325; however, because of the <, we get a base score of 2.6 instead of 0.0.

The following specifications show <= should be used:
v3.1: https://www.first.org/cvss/specification-document#7-1-Base-Metrics-Equations
v3.0: https://www.first.org/cvss/v3.0/specification-document#CVSS-v3-0-Equations

cvecache segfaults on 32 bit architectures

While attempting to package nvdtools for Fedora, I've noticed the test suite reliably segfaults on 32bit architectures (notably, i686 and armv7hl). You can see the build attempts and logs at https://koji.fedoraproject.org/koji/taskinfo?taskID=58488501

Specifically, looking at the i686 log at https://kojipkgs.fedoraproject.org//work/tasks/8539/58488539/build.log the failure is

github.com/facebookincubator/nvdtools/cmd/cpe2cve
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x565ba5ac]
goroutine 23 [running]:
runtime/internal/atomic.Xadd64(0x585c002c, 0x1, 0x0, 0x56741620, 0x5860e158)
	/usr/lib/golang/src/runtime/internal/atomic/asm_386.s:105 +0xc
github.com/facebookincubator/nvdtools/cvefeed.(*Cache).Get(0x585c0000, 0x58602030, 0x2, 0x2, 0x1, 0x58616190, 0x4c)
	/builddir/build/BUILD/nvdtools-0.1.4/_build/src/github.com/facebookincubator/nvdtools/cvefeed/cvecache.go:136 +0x3b
github.com/facebookincubator/nvdtools/cmd/cpe2cve.processAll(0x58600240, 0x58600280, 0x585c20e0, 0x4, 0x6, 0x7, 0x0, 0x0, 0x0, 0x0, ...)
	/builddir/build/BUILD/nvdtools-0.1.4/_build/src/github.com/facebookincubator/nvdtools/cmd/cpe2cve/cpe2cve.go:73 +0xccc
github.com/facebookincubator/nvdtools/cmd/cpe2cve.processInput.func1(0x58600240, 0x58600280, 0x585c20e0, 0x4, 0x6, 0x7, 0x0, 0x0, 0x0, 0x0, ...)
	/builddir/build/BUILD/nvdtools-0.1.4/_build/src/github.com/facebookincubator/nvdtools/cmd/cpe2cve/cpe2cve.go:139 +0x60
created by github.com/facebookincubator/nvdtools/cmd/cpe2cve.processInput
	/builddir/build/BUILD/nvdtools-0.1.4/_build/src/github.com/facebookincubator/nvdtools/cmd/cpe2cve/cpe2cve.go:138 +0x27f
exit status 2
FAIL	github.com/facebookincubator/nvdtools/cmd/cpe2cve	0.009s

which points to the atomic.AddInt64 at https://github.com/facebookincubator/nvdtools/blob/master/cvefeed/cvecache.go#L136

Add support for ranges on cpe2cve -e flag

In csv2cpe, when users want to exclude fields from the output, they can use -e=1,4,6-8 but in cpe2cve users have to specify each field individually. On large CSV outputs this can be tricky.

We should support commas and ranges in cpe2cve like we do in csv2cpe.

TestCacheEviction fails on 32-bit architectures.

Thanks for the fix! Unfortunately it looks like something else is amiss here. If I apply that patch I don't get the segfault anymore, but the test still fails:

      testing: github.com/facebookincubator/nvdtools
github.com/facebookincubator/nvdtools/cmd/cpe2cve
PASS
ok  	github.com/facebookincubator/nvdtools/cmd/cpe2cve	0.008s
github.com/facebookincubator/nvdtools/cmd/csv2cpe
PASS
ok  	github.com/facebookincubator/nvdtools/cmd/csv2cpe	0.005s
github.com/facebookincubator/nvdtools/cmd/redhat_filter
PASS
ok  	github.com/facebookincubator/nvdtools/cmd/redhat_filter	0.005s
github.com/facebookincubator/nvdtools/cmd/rpm2cpe
PASS
ok  	github.com/facebookincubator/nvdtools/cmd/rpm2cpe	0.004s
github.com/facebookincubator/nvdtools/cmd/wfnconvert
PASS
ok  	github.com/facebookincubator/nvdtools/cmd/wfnconvert	0.005s
github.com/facebookincubator/nvdtools/cpedict
PASS
ok  	github.com/facebookincubator/nvdtools/cpedict	0.006s
github.com/facebookincubator/nvdtools/cvefeed
--- FAIL: TestCacheEviction (0.00s)
    eviction_test.go:64: concurrent run: cache size 1758/2048; 2 records cached
    eviction_test.go:91: sequential run #1: more than 1 record cached (2)
    eviction_test.go:93: sequential run #1: cache size 2046/2048; 2 records cached
    eviction_test.go:122: sequentual run #2: cache size 1704/2048; 8 records cached
FAIL
exit status 1
FAIL	github.com/facebookincubator/nvdtools/cvefeed	0.022s

Full build output is at https://kojipkgs.fedoraproject.org//work/tasks/6747/58936747/build.log for i686, https://koji.fedoraproject.org/koji/taskinfo?taskID=58936690 is the build task with all platforms.

Originally posted by @davide125 in #167 (comment)

Invalid CVSS v2 environmental score computation

Still while fuzzing the implementation, I discovered that environmental scores were not computed properly, leading to invalid scores.
For instance, the following Go code computes the three scores and prints them.

package main

import (
	"fmt"

	"github.com/facebookincubator/nvdtools/cvss2"
)

func main() {
	vec, _ := cvss2.VectorFromString("AV:A/AC:L/Au:N/C:C/I:C/A:C/CDP:H/TD:H/CR:M/IR:L/AR:M")

	b, t, e := vec.BaseScore(), vec.TemporalScore(), vec.EnvironmentalScore()
	fmt.Printf("Scores: %.1f;%.1f;%.1f\n", b, t, e)
}

produces ->

Scores: 8.3;8.3;9.1

You can check this input is valid using the NVD calculator but then computes an environmental score of 9.1 (should be 9.0).

cvecache panics on eviction

Description

When using cpe2cve with a positive -cache_size (0 and negative work fine) a panic occurs:

panic: attempted to evict non-existent record

goroutine 12 [running]:
github.com/facebookincubator/nvdtools/cvefeed.(*Cache).evict(0xc0046dedc0, 0x0)
	/home/s/go/src/github.com/facebookincubator/nvdtools/cvefeed/cvecache.go:238 +0x12a
github.com/facebookincubator/nvdtools/cvefeed.(*Cache).Get(0xc0046dedc0, 0xc00000e5a0, 0x1, 0x1, 0x1, 0xc0059d4110, 0x10)
	/home/s/go/src/github.com/facebookincubator/nvdtools/cvefeed/cvecache.go:171 +0x425
main.processAll(0xc0059d2600, 0xc0059d2660, 0xc0046c4d20, 0x1, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	/home/s/nvdtools/cmd/cpe2cve/cpe2cve.go:73 +0xd43
main.processInput.func1(0xc0059d2600, 0xc0059d2660, 0xc0046c4d20, 0xc00008c0f0, 0xc0059d40d8, 0xc0059d40e0)
	/home/s/nvdtools/cmd/cpe2cve/cpe2cve.go:139 +0xcc
created by main.processInput
	/home/s/nvdtools/cmd/cpe2cve/cpe2cve.go:138 +0x309

To reproduce

echo "cpe:/a:foo:bar:*" | ./cpe2cve -cpe 1 -e 1 -cve 1 -cache_size 1 ./nvdcve-1.1-2002.json.gz
This bug doesn't happen when the cache size is large enough (432 for the 2002 feed).

Improper Input Validation in CVSS v3 parsing

While fuzzing this implementation, I discovered that some invalid inputs did not raise errors.
This could be categorized as CWE-20.

For instance, the following Go code does not produce any error.

package main

import (
	"fmt"

	"github.com/facebookincubator/nvdtools/cvss3"
)

func main() {
	vec, err := cvss3.VectorFromString("CVSS:3.1/AV:")

	fmt.Printf("vec: %v\n", vec)
	fmt.Printf("err: %v\n", err)
}

produces ->

vec: CVSS:3.1/
err: <nil>

You can check this input is invalid, using the official first.org calculator which does not give scores as it lacks a lot of valid info, or by looking at the specification Table 15 which shows that base metrics can't be empty.

EDIT: using version v0.1.5.

[cpe2cve] feature request: feed overrides.

At the moment, feeds consumed by cpe2cve tools are expected to have unique set of CVEs. This works fine for consuming the feed published by NVD, but does not behave properly when feeds from other vendors that have platform specification for the same CVEs are added to the mix.
The basic functionality would be to merge the LogicalTest interfaces for each CVEItem with OR operator. Ideally, we would also like to have a way to 'override' a CVE (i.e. merge it with an AND operator). This can be useful for converted vendor-specific update feeds, which usually only include the CVE itself and a patch which fixes it; applying this information with NAND operator to the main platform specification would solve the issue.

Add support to include CWE and CVSS score in the output of cpe2cve

Today, the output of cpe2cve only includes a CVE, and optionally the matched CPEs. We should have flags to optionally include the CWE and CVSS score (the base score) in the processor's output.

My suggestion is to allow -cwe=N and -cvss=N where N is the column number of the output.

Remove support for NVD CVE XML feeds

It's probably time we start discussing deprecating support for CVE XML feeds. We've been using NVD CVE JSON 1.0 for half year and the benefits from versions ranges are satisfactory.

In the codebase, however, having XML around makes things a log more complex, including manual edits to the auto-generated JSON schema to support the "nvdcommon" package which merges both XML and JSON into a common interface.

As we add more converters (the so called $vendor2nvd) to this codebase, we will need a proper generated JSON schema that is not edited manually, ever.

The only breaking change I can see is for the cpe2cve -feed flag, which should probably be removed as well if we make JSON the one and only supported feed.

Use with Debian based packages

Hi, and thanks for an interesting project.

Whilst my servers are all rpm package based, all my engineers use Debian based distros. I've had a play with dpkg to try and emulate rpm output. However I get a significant number of misses. e.g.

dpkg-query --show -f='${binary:Package}-${Version}.${Architecture}\n' | rpm2cpe -rpm=1 -cpe=2

appears to give valid results e.g.:

wamerican-2018.04.16-1.all      cpe:/a::wamerican:2018.04.16:1:~~~~all~                                                                                                                         
wbritish-2018.04.16-1.all       cpe:/a::wbritish:2018.04.16:1:~~~~all~                                                                                                                          
wget-1.20.3-1ubuntu1.amd64      cpe:/a::wget:1.20.3:1ubuntu1:~~~~amd64~                                                                                                                         
whiptail-0.52.21-4ubuntu2.amd64 cpe:/a::whiptail:0.52.21:4ubuntu2:~~~~amd64~                                                                                                                    
whoopsie-preferences-22.amd64   cpe:/a::whoopsie:preferences:22:~~~~amd64~

but also a lot of errors e.g.:

rpm2cpe: error: couldn't process record []: couldn't parse RPM name from field "zoom-5.4.54779.1115.amd64": can't get fields from "zoom-5.4.54779.1115.amd64": can't find version in pkg "zoom"
rpm2cpe: error: couldn't process record []: couldn't parse RPM name from field "whois-5.5.6.amd64": can't get fields from "whois-5.5.6.amd64": can't find versi
on in pkg "whois"                                                                                                                                                                               
rpm2cpe: error: couldn't process record []: couldn't parse RPM name from field "whoopsie-0.2.69ubuntu0.3.amd64": can't get fields from "whoopsie-0.2.69ubuntu0.3.amd64": can't find version in p
kg "whoopsie"                                                                                                                                                                                   
rpm2cpe: error: couldn't process record []: couldn't parse RPM name from field "xorg-1:7.7+19ubuntu14.amd64": can't get fields from "xorg-1:7.7+19ubuntu14.amd64": can't find version in pkg "xo
rg"

I could clean up the dpkg output to remove the errors, but that will mean missing substantial numbers of packages. Any suggestions on how I perhaps modify my one-liner? Or is there a better method to achieve what I want?

My eventual aim is to install a cron on all the engineer's development machines to input their stack to a central database and run the CVE checks for them alongside doing the same for the servers. Any help appreciated.

Panic in csv2cpe

Off by one error :)

fiorix@devvm1527 ~/tmp/apt-cve $ echo a,b,c | csv2cpe -cpe_part=1 -cpe_product=2 -cpe_product=3 -e=1-3
cpe:/a::c
fiorix@devvm1527 ~/tmp/apt-cve $ echo a,b,c | csv2cpe -cpe_part=1 -cpe_product=2 -cpe_product=4 -e=1-3
panic: runtime error: index out of range

goroutine 1 [running]:
main.(*AttributeColumnMap).CPE(0xc000020120, 0xc0000ae240, 0x3, 0x3, 0x0, 0x0, 0xc000075c28, 0xc000075c30, 0x40c068)
        third-party-source/go/github.com/facebookincubator/nvdtools/cmd/csv2cpe/csv2cpe.go:187 +0x86e
main.(*Processor).Process(0xc000075f38, 0xc000020120, 0x4f00a0, 0xc00000e010, 0x4f00c0, 0xc00000e018, 0x0, 0x0)
        third-party-source/go/github.com/facebookincubator/nvdtools/cmd/csv2cpe/csv2cpe.go:115 +0x237
main.main()
        third-party-source/go/github.com/facebookincubator/nvdtools/cmd/csv2cpe/csv2cpe.go:76 +0x39d
fiorix@devvm1527 ~/tmp/apt-cve $ echo a,b,c | csv2cpe -cpe_part=1 -cpe_product=2 -cpe_product=5 -e=1-3
cpe:/a

nvdjson.smartVerCmp is bugged.

It only compares the first part of the version (before the first dot) and even crashes on some versions (e.g. "10.21" vs "10.23")

Improper Input Validation in CVSS v2 parsing

While fuzzing this implementation, I discovered that some invalid inputs did not raise errors.
This could be categorized as CWE-20.

For instance, the following Go code does not produce any error.

package main

import (
	"fmt"

	"github.com/facebookincubator/nvdtools/cvss2"
)

func main() {
	vec, err := cvss2.VectorFromString("AV:A/AC:L/Au:N/C:C/I:C/A:C/CDP:H/TD:H/CR:H")

	fmt.Printf("vec: %v\n", vec)
	fmt.Printf("err: %v\n", err)
}

produces ->

vec: (AV:A/AC:L/Au:N/C:C/I:C/A:C/CDP:H/TD:H/CR:H)
err: <nil>

You can check this input is invalid, using the NVD calculator which displays the vector with all environmental metrics when given this vector, or by looking at the specification Table 13 which shows the environmental vector must contain all metrics when in the CVSS vector.

Found using version/tag v0.1.5.

SyncError as return value

In the Do function in sync.go, SyncError is returned as an error value, and that value is later tested for nil. This is a bug, as a nil value returned as error (which is an interface) will produce an interface object with nil as the value, and SyncError as the type, causing the caller to mistakenly classify it as an error.

CVE cache grows indefinitely

The cache code from cvefeed is not an LRU and does not have a purge option. When used in (thrift) servers, all client queries are cached until it OOMs.

We should be able to define at least the maximum size for the cache.

Issues installing on Raspbian -

I've tried install tools on Raspbian (Raspbian GNU/Linux 10 (buster)) with go version

go version go1.11.6 linux/arm

This was installed from the repositories on Raspbian. I'll try installing a newer version.

It's a fresh go installation, on a recently installed RPI.

go get github.com/facebookincubator/nvdtools/...                                                                                       
# github.com/facebookincubator/nvdtools/providers/lib/client                                                                                          
go/src/github.com/facebookincubator/nvdtools/providers/lib/client/client.go:36:14: undefined: http.NewRequestWithContext                              
go/src/github.com/facebookincubator/nvdtools/providers/lib/client/debug.go:73:23: req.Header.Clone undefined (type http.Header has no field or method Clone)      
# github.com/facebookincubator/nvdtools/cvefeed                                                                                                       
go/src/github.com/facebookincubator/nvdtools/cvefeed/diff.go:172:17: invalid operation: 1 << i (shift count type int, must be unsigned integer)       
go/src/github.com/facebookincubator/nvdtools/cvefeed/diff.go:175:33: invalid operation: 1 << i (shift count type int, must be unsigned integer)       
go/src/github.com/facebookincubator/nvdtools/cvefeed/diff.go:176:33: invalid operation: 1 << i (shift count type int, must be unsigned integer)       
# github.com/spf13/cobra                                                                                                                              
go/src/github.com/spf13/cobra/bash_completions.go:22:24: undefined: io.StringWriter                                                                   
go/src/github.com/spf13/cobra/bash_completions.go:383:26: undefined: io.StringWriter                                                                  
go/src/github.com/spf13/cobra/bash_completions.go:425:24: undefined: io.StringWriter                                                                  
go/src/github.com/spf13/cobra/bash_completions.go:437:27: undefined: io.StringWriter                                                                  
go/src/github.com/spf13/cobra/bash_completions.go:475:25: undefined: io.StringWriter                                                                  
go/src/github.com/spf13/cobra/bash_completions.go:486:20: undefined: io.StringWriter                                                                  
go/src/github.com/spf13/cobra/bash_completions.go:501:38: undefined: io.StringWriter                                                                  
go/src/github.com/spf13/cobra/bash_completions.go:527:21: undefined: io.StringWriter                                                                  
go/src/github.com/spf13/cobra/bash_completions.go:564:28: undefined: io.StringWriter                                                                  
go/src/github.com/spf13/cobra/bash_completions.go:589:29: undefined: io.StringWriter                                                                  
go/src/github.com/spf13/cobra/bash_completions.go:589:29: too many errors                                                                             
# github.com/go-sql-driver/mysql                                                                                                                      
go/src/github.com/go-sql-driver/mysql/nulltime.go:36:15: undefined: sql.NullTime
go env                                                                                                                          
GOARCH="arm"                                                                                                                                          
GOBIN=""                                                                                                                                              
GOCACHE="/home/user1/.cache/go-build"                                                                                                                
GOEXE=""                                                                                                                                              
GOFLAGS=""                                                                                                                                            
GOHOSTARCH="arm"                                                                                                                                      
GOHOSTOS="linux"                                                                                                                                      
GOOS="linux"                                                                                                                                          
GOPATH="/home/user1/go"                                                                                                                              
GOPROXY=""                                                                                                                                            
GORACE=""                                                                                                                                             
GOROOT="/usr/lib/go-1.11"                                                                                                                             
GOTMPDIR=""                                                                                                                                           
GOTOOLDIR="/usr/lib/go-1.11/pkg/tool/linux_arm"                                                                                                       
GCCGO="gccgo"                                                                                                                                         
GOARM="6"                                                                                                                                             
CC="gcc"                                                                                                                                              
CXX="g++"                                                                                                                                             
CGO_ENABLED="1"                                                                                                                                       
GOMOD=""                                                                                                                                              
CGO_CFLAGS="-g -O2"                                                                                                                                   
CGO_CPPFLAGS=""                                                                                                                                       
CGO_CXXFLAGS="-g -O2"                                                                                                                                 
CGO_FFLAGS="-g -O2"                                                                                                                                   
CGO_LDFLAGS="-g -O2"                                                                                                                                  
PKG_CONFIG="pkg-config"                                                                                                                               
GOGCCFLAGS="-fPIC -marm -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build679389298=/tmp/go-build -gno-record-gcc-switches"

Trying to install and compile within a container I've no issues in the RPI, but based on the golang image that would be with version go version go1.16.5 linux/arm

wrong CPE matching

Some otherwise valid CPE identifiers produce wrong matchings.

As an example, this match is good :

$ echo 'cpe:2.3:a:clamav:clamav:1.0.0:*:*:*:*:*:*:*' | cpe2cve -cpe=1 -cve=2 ./nvd/nvdcve-1.1-*.json.gz
cpe:2.3:a:clamav:clamav:1.0.0:*:*:*:*:*:*:*     CVE-2023-20032
cpe:2.3:a:clamav:clamav:1.0.0:*:*:*:*:*:*:*     CVE-2023-20052

But this one matches CVE-2021-45967 which has nothing to do with clamav :

$ echo 'cpe:2.3:a:*:clamav:1.0.0:*:*:*:*:*:*:*' | cpe2cve -cpe=1 -cve=2 ./nvd/nvdcve-1.1-*.json.gz
cpe:2.3:a:*:clamav:1.0.0:*:*:*:*:*:*:*  CVE-2023-20052
cpe:2.3:a:*:clamav:1.0.0:*:*:*:*:*:*:*  CVE-2021-45967
cpe:2.3:a:*:clamav:1.0.0:*:*:*:*:*:*:*  CVE-2023-20032

Search by CPE with "running with" section

Hello!

There is CVE-2020-16022 for google:chrome with section "Running on/with".

cpe2cve does not find it by query:

$ echo echo cpe:/a:google:chrome:83.0.4103.61 | cpe2cve -cpe=1 -cve=2 *2020*.json.gz | grep 16022

How to find CVEs marked with "running with" configuration?

If you try this:

$ echo cpe:/a:google:chrome,cpe:/o:linux:linux_kernel:- | cpe2cve -cpe=1 -cve=2 feeds/nvd/*2020*.json.gz | grep 35508
cpe:/a:google:chrome,cpe:/o:linux:linux_kernel:-        CVE-2020-35508

It gives you CVE-2020-35508, which has no relation to a:google:chrome.

Creation of a web API to use nvdtools as a vulnerability monitoring service.

Hello,

I work for the group LINAGORA, and we have the project LinInfoSec to build a notification tool to monitor CVE publications relating to a given list of software, and integrate it with our ticketing system SmartSLA.
To bootstrap our project, we would like to base it on an existing OSS project and nvdtools is a great candidate.

If we do so, we would add some functionality to the project:

  • A web API including:
    • Efficient search for the CPE dictionary
    • Configuring a set of CPE URIs to be monitored for new CVEs
  • Automatic fetching of recent CVE publications
  • A notification system when a new CVE matches the mentioned set of CPEs

We would implement these as a separate binary, then use this API to provide additionnal functionnality in our product SmartSLA.

We could fork the project to implement these features on our own, regardless of your long term objectives for nvdtools, but we much rather collaborate with you so that our contributions can be merged into nvdtools.
Would you be interested to collaborate with us on such a project? If you are, we could meet through video conference to discuss it.

Thanks,
Sosthène Guédon for LINAGORA

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.