Code Monkey home page Code Monkey logo

cve-search's Introduction

cve-search

Join the chat at https://gitter.im/cve-search/cve-search Build & Test Black formatting CodeQL

cve-search logo

cve-search is a tool to import CVE (Common Vulnerabilities and Exposures) and CPE (Common Platform Enumeration) into a MongoDB to facilitate search and processing of CVEs.

The main objective of the software is to avoid doing direct and public lookups into the public CVE databases. Local lookups are usually faster and you can limit your sensitive queries via the Internet.

cve-search includes a back-end to store vulnerabilities and related information, an intuitive web interface for search and managing vulnerabilities, a series of tools to query the system and a web API interface.

cve-search is used by many organizations including the public CVE services of CIRCL.

This document gives you basic information how to start with cve-search. For more information please refer to the documentation in the /doc folder of this project.

Getting started

Check the documentation to get you started

Usage

You can search the database using search.py.

usage: search.py [-h] [-q Q] [-p P [P ...]] [--only-if-vulnerable] [--strict_vendor_product] [--lax] [-f F] [-c C] [-o O]
                 [-l] [-n] [-r] [-a] [-v V] [-s S] [-t T] [-i I]

Search for vulnerabilities in the National Vulnerability DB. Data from http://nvd.nist.org.

options:
  -h, --help            show this help message and exit
  -q Q                  Q = search pip requirements file for CVEs, e.g. dep/myreq.txt
  -p P [P ...]          S = search one or more products, e.g. o:microsoft:windows_7 or o:cisco:ios:12.1 or
                        o:microsoft:windows_7 o:cisco:ios:12.1. Add --only-if-vulnerable if only vulnerabilities that
                        directly affect the product are wanted.
  --only-if-vulnerable  With this option, "-p" will only return vulnerabilities directly assigned to the product. I.e.
                        it will not consider "windows_7" if it is only mentioned as affected OS in an adobe:reader
                        vulnerability.
  --strict_vendor_product
                        With this option, a strict vendor product search is executed. The values in "-p" should be
                        formatted as vendor:product, e.g. microsoft:windows_7
  --lax                 Strict search for software version is disabled. Likely gives false positives for earlier
                        versions that were not yet vulnerable. Note that version comparison for non-numeric values
                        is done with simplifications.
  -f F                  F = free text search in vulnerability summary
  -c C                  search one or more CVE-ID
  -o O                  O = output format [csv|html|json|xml|cveid]
  -l                    sort in descending mode
  -n                    lookup complete cpe (Common Platform Enumeration) name for vulnerable configuration
  -r                    lookup ranking of vulnerable configuration
  -a                    Lookup CAPEC for related CWE weaknesses
  -v V                  vendor name to lookup in reference URLs
  -s S                  search in summary text
  -t T                  search in last n day
  -i I                  Limit output to n elements (default: unlimited)

Examples:

./bin/search.py -p cisco:ios:12.4
./bin/search.py -p cisco:ios:12.4 -o json
./bin/search.py -f nagios -n
./bin/search.py -p microsoft:windows_7 -o html

If you want to search all the WebEx vulnerabilities and only printing the official references from the supplier.

./bin/search.py -p webex: -o csv  -v "cisco"

You can also dump the JSON for a specific CVE ID.

./bin/search.py -c CVE-2010-3333 -o json

Or dump the last 2 CVE entries in RSS or Atom format.

./bin/dump_last.py -f atom -l 2

Or you can use the webinterface.

./web/index.py

Usage of the ranking database

There is a ranking database allowing to rank software vulnerabilities based on their common platform enumeration name. The ranking can be done per organization or department within your organization or any meaningful name for you.

As an example, you can add a partial CPE name like "sap:netweaver" which is very critical for your accounting department.

./sbin/db_ranking.py  -c "sap:netweaver" -g "accounting" -r 3

and then you can lookup the ranking (-r option) for a specific CVE-ID:

./bin/search.py -c CVE-2012-4341  -r  -n

Advanced usage

As cve-search is based on a set of tools, it can be used and combined with standard Unix tools. If you ever wonder what are the top vendors using the term "unknown" for their vulnerabilities:

python3 bin/search_fulltext.py -q unknown -f \
    | jq -c '. | .vulnerable_configuration[0]' \
    | cut -f5 -d: | sort  | uniq -c  | sort -nr | head -10

1500 oracle
381 sun
372 hp
232 google
208 ibm
126 mozilla
103 microsoft
100 adobe
 78 apple
 68 linux

You can compare CVSS (Common Vulnerability Scoring System ) values of some products based on their CPE name. Like comparing oracle:java versus sun:jre and using R to make some statistics about their CVSS values:

python3 bin/search.py -p oracle:java -o json \
  | jq -r '.cvss' | Rscript -e 'summary(as.numeric(read.table(file("stdin"))[,1]))'

Min. 1st Qu.  Median    Mean 3rd Qu.    Max.
1.800   5.350   9.300   7.832  10.000  10.000
python3 bin/search.py -p sun:jre -o json \
  | jq -r '.cvss' | Rscript -e 'summary(as.numeric(read.table(file("stdin"))[,1]))'

Min. 1st Qu.  Median    Mean 3rd Qu.    Max.
0.000   5.000   7.500   7.333  10.000  10.000

Fulltext indexing

If you want to index all the CVEs from your current MongoDB collection:

./sbin/db_fulltext.py -l 0

and you query the fulltext index (to get a list of matching CVE-ID):

./bin/search_fulltext.py -q NFS -q Linux

or to query the fulltext index and output the JSON object for each CVE-ID:

./bin/search_fulltext.py -q NFS -q Linux -f

Fulltext visualization

The fulltext indexer visualization is using the fulltext indexes to build a list of the most common keywords used in CVE. NLTK is required to generate the keywords with the most common English stopwords and lemmatize the output. NTLK for Python 3 exists but you need to use the alpha version of NLTK.

./bin/search_fulltext.py  -g -s >cve.json

cve-search visualization

You can see a visualization on the demo site.

Web interface

The web interface is a minimal interface to see the last CVE entries and query a specific CVE. You'll need flask in order to run the website and Flask-PyMongo. To start the web interface:

cd ./web
./index.py

Then you can connect on http://127.0.0.1:5000/ to browser the last CVE.

Web API interface

The web interface includes a minimal JSON API to get CVE by ID, by vendor or product. A public version of the API is also accessible on cve.circl.lu.

List the know vendors in JSON

curl "http://127.0.0.1:5000/api/browse/"

Dump the product of a specific vendor in JSON

curl "http://127.0.0.1:5000/api/browse/zyxel"
{
  "product": [
    "n300_netusb_nbg-419n",
    "n300_netusb_nbg-419n_firmware",
    "p-660h-61",
    "p-660h-63",
    "p-660h-67",
    "p-660h-d1",
    "p-660h-d3",
    "p-660h-t1",
    "p-660h-t3",
    "p-660hw",
    "p-660hw_d1",
    "p-660hw_d3",
    "p-660hw_t3"
  ],
  "vendor": "zyxel"
}

Find the associated vulnerabilities to a vendor and a product.

curl "http://127.0.0.1:5000/api/search/zyxel/p-660hw" | jq .
[
  {
    "cwe": "CWE-352",
    "references": [
      "http://www.exploit-db.com/exploits/33518",
      "http://secunia.com/advisories/58513",
      "http://packetstormsecurity.com/files/126812/Zyxel-P-660HW-T1-Cross-Site-Request-Forgery.html",
      "http://osvdb.org/show/osvdb/107449"
    ],
    "vulnerable_configuration": [
      "cpe:/h:zyxel:p-660hw:_t1:v3"
    ],
    "Published": "2014-06-16T14:55:09.713-04:00",
    "id": "CVE-2014-4162",
    "Modified": "2014-07-17T01:07:29.683-04:00",
    "cvss": 6.8,
    "summary": "Multiple cross-site request forgery (CSRF) vulnerabilities in the Zyxel P-660HW-T1 (v3) wireless router allow remote attackers to hijack the authentication of administrators for requests that change the (1) wifi password or (2) SSID via a request to Forms/WLAN_General_1."
  },
  {
    "cwe": "CWE-20",
    "references": [
      "http://www.kb.cert.org/vuls/id/893726"
    ],
    "vulnerable_configuration": [
      "cpe:/h:zyxel:p-660h-63:-",
      "cpe:/h:zyxel:p-660h-t1:-",
      "cpe:/h:zyxel:p-660h-d3:-",
      "cpe:/h:zyxel:p-660h-t3:v2",
      "cpe:/h:zyxel:p-660h-t1:v2",
      "cpe:/h:zyxel:p-660h-d1:-",
      "cpe:/h:zyxel:p-660h-67:-",
      "cpe:/h:zyxel:p-660h-61:-",
      "cpe:/h:zyxel:p-660hw_t3:v2",
      "cpe:/h:zyxel:p-660hw_t3:-",
      "cpe:/h:zyxel:p-660hw_d3:-",
      "cpe:/h:zyxel:p-660hw_d1:v2",
      "cpe:/h:zyxel:p-660hw_d1:-",
      "cpe:/h:zyxel:p-660hw:_t1:v2",
      "cpe:/h:zyxel:p-660hw:_t1:-"
    ],

Software using cve-search

Docker versions

Official dockerized version of cve-search:

CVE-Search-Docker

There are some unofficial dockerized versions of cve-search (which are not maintained by us):

Changelog

You can find the changelog on GitHub Releases (legacy changelog).

License

cve-search is free software released under the "GNU Affero General Public License v3.0"

Copyright (c) 2012 Wim Remes - https://github.com/wimremes/
Copyright (c) 2012-2024 Alexandre Dulaunoy - https://github.com/adulau/
Copyright (c) 2015-2019 Pieter-Jan Moreels - https://github.com/pidgeyl/
Copyright (c) 2020-2024 Paul Tikken - https://github.com/P-T-I

cve-search's People

Contributors

adulau avatar azobec avatar baonq-me avatar chervaliery avatar criimbow avatar dbarzin avatar dependabot[bot] avatar docarmorytech avatar eengelking avatar fafnerkeyzee avatar galaxygamingboy avatar hack3r-0m avatar hashworks avatar igama avatar irootgeek avatar janidetiger avatar kairis avatar lvets avatar m0dex avatar mdeous avatar noraj avatar oh2fih avatar p-t-i avatar pidgeyl avatar pombredanne avatar psychedelys avatar rafiot avatar roccovanasselt avatar timeemit avatar wimremes avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

cve-search's Issues

cve_doc.py Impact Error in some CVE

Hi!
I have a problem.
I Run this command

python3 cve_doc.py -c CVE-2015-0997 | asciidoctor - >test.html

and get an error

Traceback (most recent call last):
  File "cve_doc.py", line 47, in <module>
    if cve['impact']:
KeyError: 'impact'

But i run a another CVE, work OK (Html perfect)

python3 cve_doc.py -c CVE-2015-0003 | asciidoctor - >test.html

This error is also present in other CVE, for example CVE-2016-0997 and CVE-2016-0997

Why it works only in some CVE?

Greetings.

image

db_update.py UnicodeDecodeError

Tried to update my db and getting errors

Did a git pull after the first time i saw this and its still getting the same error

python3 db_updater.py -v
Not modified
Not modified
Not modified
Not modified
Not modified
Not modified
Not modified
Not modified
Not modified
[+]Success to create index id on cpe
[+]Success to create index id on cpeother
[+]Success to create index id on cves
[+]Success to create index vulnerable_configuration on cves
[+]Success to create index Modified on cves
[+]Success to create index [('summary', 'text')] on cves
[+]Success to create index id on vfeed
[+]Success to create index id on vendor
[+]Success to create index id on d2sec
[+]Success to create index id on mgmt_whitelist
[+]Success to create index id on mgmt_blacklist
[+]Success to create index related_weakness on capec 
Starting cves
cves has 72400 elements (0 update)
Starting cpe
cpe has 106092 elements (0 update)
Starting cpeother
cpeother has 252334 elements (0 update)
Starting vfeed
vfeed has 73146 elements (0 update)
Starting vendor
vendor has 1406 elements (0 update) 
Starting cwe 
Traceback (most recent call last):
 File "/root/cve-search/sbin/db_mgmt_cwe.py", line 107, in <module> 
parser.parse(f)
File "/usr/lib/python3.4/xml/sax/expatreader.py", line 107, in parse
xmlreader.IncrementalParser.parse(self, source)
 File "/usr/lib/python3.4/xml/sax/xmlreader.py", line 124, in parse
 buffer = file.read(self._bufsize)
File "/usr/lib/python3.4/encodings/ascii.py", line 26, in decode
return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 46797: ordinal not in range(128)
cwe has 0 elements (0 update)
Starting capec
capec has 463 elements (0 update)
Starting d2sec
d2sec has 246 elements (0 update)
Starting ms
ms has 1345 elements (0 update)
Starting redis-nist-ref
redis-nist-ref has 0 elements (0 update)
Starting ensureindex

A different result with two commands in search_fulltext.py

Hello there!
Run the command that is on the sheet 10 PDF brucon2015-cve-search.pdf and it works perfect.

python3 ./bin/search_fulltext.py -q unknown -f | jq -c '. | . vulnerable_configuration[0]' | cut -f5 -d: | sort | uniq -c | sort -nr | head -10

Result:

   1965 oracle
    317 hp
    283 google
    246 sun
    135 mozilla
    116 ibm
     83 adobe
     35 mysql
     31 microsoft
     25 novell

But what I want to do is to that command, adding a date search.
I use this command:

python3 ./bin/search_fulltext.py -q unknown -f | jq -c '. 'vulnerable_configuration[0]' and ((.Modified >= "2016-01-01") and (.Modified <= "2016-01-30"))' | cut -f5 -d: | sort | uniq -c | sort -nr | head -10

The result is different from the first command

   4078 false
      5 true

I knew the command and ((.Modified >= "2016-01-01") through issue #116

What am I doing wrong?

Gracias!

{ and } in search_fulltext.py

Hello there!
If it's not a bug, I apologize.

But if I run the command that is written in "brucon2015-cve-search.pdf", page 10

python3 ./bin/search_fulltext.py -q unknown -f | jq -r '. | . vulnerable_configuration[0]' | cut -f5 -d: | sort | uniq -c | sort -nr | head -10

The result is:

   4078 }
   4078 {
   3930 oracle
    634 hp
    564 google
    492 sun
    270 mozilla
    224 ibm
    166 adobe
     70 mysql

As you can see, the first two results are {and} when it should not be.

This also occurs in the example on page 11.

python3 ./bin/search_fulltext.py -q unknown -f | jq -r '. | . vulnerable_configuration[0]' | cut -f5,6 -d: | sort | uniq -c | sort -nr | head -10

   4078 }
   4078 {
    504 oracle:database_server
    498 google:chrome
    388 oracle:fusion_middleware
    356 oracle:jre
    318 oracle:e-business_suite
    298 oracle:mysql
    226 oracle:jdk
    216 sun:jre

Any idea how to fix it?

Thanks!

edit:
Screenshot:
image

Bulk update - issue with update on vendor database

Starting vendor
Traceback (most recent call last):                           ] 0/1440
  File ".../cve-search/db_mgmt_vendorstatements.py", line 83, in <module>
    bulk.find({'id': statement['id']}).upsert().update({'id': statement['id']}, {"$set":{'statement': statement['statement'], 'id': statement['id'], 'organization': statement['organization'], 'contributor': statement['contributor'], 'lastmodified': statement['lastmodified']}})
TypeError: update() takes 2 positional arguments but 3 were given

Get partial CVSS score in the database as well

I am concidering importing the partial cvss score (impact and exploit) into the database. It is not in our NVD Source list, but it is in vFeed.
I would like a second opinion on the implementation of this:

  • Do I add this information to the collection cves (so adding to that data) or to vFeed? These scores are part of the cve, but I think we should not tamper with that list. Adding it to vFeed will separate this, but will create an item in the vFeed database for every single CVE. This score will also not be available then, when you do a CVE lookup without vFeed data
  • When I implement it, do I group these in sub documents, like I did with all the vFeed data? If I add this to the cves collection, do I group CVSS-Base, CVSS-Impact, CVSS-Exploit and CVSS-Time under a document CVSS?

@adulau @wimremes

Structure

Should we put the query interfaces (XMPP bot, IRC bot, API, searcy.py, dump_last.py, ...) in a different folder or not? Would it be a good idea to split database scripts (back end) from the interfaces (front end)?

XMPP issue?

I found a weird issue with the XMPP client. When I use a command that accesses the API, I get no return. I put some prints in there to see where it stops, and I couldn't quite find it. The api searcher returns the CVE (when i use get) all the way up to def message, but I get no return. I'm using ejabberd. Can you reproduce this issue? @adulau

db_mgmt_cpe_dictionary.py execution takes a long time\not ending

Hi,
as part of the setup i'm executing db_mgmt_cpe_dictionary.py, which runs for a very long time but doesnt complete. i've let it run for over 3 hours, but still nothing.
when i ctrl-c the execution this is the output.
githubissue

should i just let it run longer ? or is there something not working here ?
Thanks,

potential bug in ./sbin/db_mgmt_d2sec.py

./sbin/db_mgmt_d2sec.py threw the following error:

Traceback (most recent call last):
  File "/home/PidgeyL/git/PidgeyL/cve-search/sbin/db_mgmt_d2sec.py", line 109, in <module>
    parser.parse(f)
  File "/usr/lib/python3.4/xml/sax/expatreader.py", line 107, in parse
    xmlreader.IncrementalParser.parse(self, source)
  File "/usr/lib/python3.4/xml/sax/xmlreader.py", line 123, in parse
    self.feed(buffer)
  File "/usr/lib/python3.4/xml/sax/expatreader.py", line 207, in feed
    self._parser.Parse(data, isFinal)
  File "../Modules/pyexpat.c", line 459, in EndElement
  File "/usr/lib/python3.4/xml/sax/expatreader.py", line 307, in end_element
    self._cont_handler.endElement(name)
  File "/home/PidgeyL/git/PidgeyL/cve-search/sbin/db_mgmt_d2sec.py", line 74, in endElement
    if self.cveref != "":
AttributeError: 'ExploitHandler' object has no attribute 'cveref'

Adding OVAL to database?

Should we add OVAL (Open Vulnerability and Assessment Language)information to the database?
(http://oval.mitre.org/)

An example can be found below. Your thoughts?
@adulau @wimremes

    <definition id="oval:org.mitre.oval:def:9995" version="5" class="vulnerability">
      <metadata>
        <title>The Linux kernel before 2.6.16.9 and the FreeBSD kernel, when running on AMD64 and other 7th and 8th generation AuthenticAMD processors, only save/restore the FOP, FIP, and FDP x87 registers in FXSAVE/FXRSTOR when an exception is pending, which allows one process to determine portions of the state of floating point instructions of other processes, which can be leveraged to obtain sensitive information such as cryptographic keys.  NOTE: this is the documented behavior of AMD64 processors, but it is inconsistent with Intel processers in a security-relevant fashion that was not addressed by the kernels.</title>
        <affected family="unix">
          <platform>Red Hat Enterprise Linux 3</platform>
          <platform>CentOS Linux 3</platform>
          <platform>Red Hat Enterprise Linux 4</platform>
          <platform>CentOS Linux 4</platform>
          <platform>Oracle Linux 4</platform>
        </affected>
        <reference source="CVE" ref_id="CVE-2006-1056" ref_url="http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2006-1056"/>
        <description>The Linux kernel before 2.6.16.9 and the FreeBSD kernel, when running on AMD64 and other 7th and 8th generation AuthenticAMD processors, only save/restore the FOP, FIP, and FDP x87 registers in FXSAVE/FXRSTOR when an exception is pending, which allows one process to determine portions of the state of floating point instructions of other processes, which can be leveraged to obtain sensitive information such as cryptographic keys.  NOTE: this is the documented behavior of AMD64 processors, but it is inconsistent with Intel processers in a security-relevant fashion that was not addressed by the kernels.</description>
        <oval_repository>
          <dates>
            <submitted date="2010-07-09T03:56:16-04:00">
              <contributor organization="SCAP.com, LLC">Aharon Chernin</contributor>
            </submitted>
            <status_change date="2010-07-28T14:25:05.980-04:00">DRAFT</status_change>
            <status_change date="2010-08-16T04:15:26.348-04:00">INTERIM</status_change>
            <status_change date="2010-09-06T04:16:35.189-04:00">ACCEPTED</status_change>
            <modified comment="EDITED oval:org.mitre.oval:def:9995 - Expanded the vulnerability checks for RHEL 3, 4, and 5 to cover  CentOS 3, 4, 5 and Oracle Linux 4 and 5" date="2013-04-10T16:24:00.823-04:00">
              <contributor organization="G2, Inc.">Dragos Prisaca</contributor>
            </modified>
            <status_change date="2013-04-10T16:30:25.620-04:00">INTERIM</status_change>
            <status_change date="2013-04-29T04:23:59.785-04:00">ACCEPTED</status_change>
          </dates>
          <status>ACCEPTED</status>
        </oval_repository>
      </metadata>
      <criteria operator="OR">
        <criteria operator="AND" comment="OS Section: RHEL3, CentOS3">
          <criteria operator="OR" comment="RHEL3 or CentOS3">
            <extend_definition comment="The operating system installed on the system is Red Hat Enterprise Linux 3" definition_ref="oval:org.mitre.oval:def:11782"/>
            <extend_definition comment="CentOS Linux 3.x" definition_ref="oval:org.mitre.oval:def:16651"/>
          </criteria>
          <criteria operator="OR" comment="Configuration section">
            <criterion comment="kernel-BOOT is earlier than 0:2.4.21-47.EL" test_ref="oval:org.mitre.oval:tst:32158"/>
            <criterion comment="kernel-unsupported is earlier than 0:2.4.21-47.EL" test_ref="oval:org.mitre.oval:tst:32589"/>
            <criterion comment="kernel-smp-unsupported is earlier than 0:2.4.21-47.EL" test_ref="oval:org.mitre.oval:tst:32704"/>
            <criterion comment="kernel-hugemem-unsupported is earlier than 0:2.4.21-47.EL" test_ref="oval:org.mitre.oval:tst:32562"/>
            <criterion comment="kernel-hugemem is earlier than 0:2.4.21-47.EL" test_ref="oval:org.mitre.oval:tst:32078"/>
            <criterion comment="kernel is earlier than 0:2.4.21-47.EL" test_ref="oval:org.mitre.oval:tst:32513"/>
            <criterion comment="kernel-source is earlier than 0:2.4.21-47.EL" test_ref="oval:org.mitre.oval:tst:32231"/>
            <criterion comment="kernel-doc is earlier than 0:2.4.21-47.EL" test_ref="oval:org.mitre.oval:tst:32097"/>
            <criterion comment="kernel-smp is earlier than 0:2.4.21-47.EL" test_ref="oval:org.mitre.oval:tst:32708"/>
          </criteria>
        </criteria>
        <criteria operator="AND" comment="OS Section: RHEL4, CentOS4, Oracle Linux 4">
          <criteria operator="OR" comment="RHEL4, CentOS4 or Oracle Linux 4">
            <extend_definition comment="The operating system installed on the system is Red Hat Enterprise Linux 4" definition_ref="oval:org.mitre.oval:def:11831"/>
            <extend_definition comment="CentOS Linux 4.x" definition_ref="oval:org.mitre.oval:def:16636"/>
            <extend_definition comment="Oracle Linux 4.x" definition_ref="oval:org.mitre.oval:def:15990"/>
          </criteria>
          <criteria operator="OR" comment="Configuration section">
            <criterion comment="kernel-hugemem is earlier than 0:2.6.9-42.EL" test_ref="oval:org.mitre.oval:tst:32335"/>
            <criterion comment="kernel-hugemem-devel is earlier than 0:2.6.9-42.EL" test_ref="oval:org.mitre.oval:tst:32833"/>
            <criterion comment="kernel-smp-devel is earlier than 0:2.6.9-42.EL" test_ref="oval:org.mitre.oval:tst:32825"/>
            <criterion comment="kernel-largesmp-devel is earlier than 0:2.6.9-42.EL" test_ref="oval:org.mitre.oval:tst:32836"/>
            <criterion comment="kernel is earlier than 0:2.6.9-42.EL" test_ref="oval:org.mitre.oval:tst:32736"/>
            <criterion comment="kernel-devel is earlier than 0:2.6.9-42.EL" test_ref="oval:org.mitre.oval:tst:31931"/>
            <criterion comment="kernel-doc is earlier than 0:2.6.9-42.EL" test_ref="oval:org.mitre.oval:tst:32361"/>
            <criterion comment="kernel-largesmp is earlier than 0:2.6.9-42.EL" test_ref="oval:org.mitre.oval:tst:32793"/>
            <criterion comment="kernel-smp is earlier than 0:2.6.9-42.EL" test_ref="oval:org.mitre.oval:tst:32795"/>
          </criteria>
        </criteria>
      </criteria>
    </definition>

Initial import issue

Received a report from a user having an issue:

python3 ./db_mgmt.py -p

Database population started

Year 2002 imported.

Traceback (most recent call last):

  File "./db_mgmt.py", line 186, in <module>

    ret = collection.insert(ch.cves)

  File "/usr/local/lib/python3.4/site-packages/pymongo/collection.py", line 410, in insert

    _check_write_command_response(results)

  File "/usr/local/lib/python3.4/site-packages/pymongo/helpers.py", line 198, in _check_write_command_response

    raise DuplicateKeyError(error.get("errmsg"), 11000, error)

pymongo.errors.DuplicateKeyError: insertDocument :: caused by :: 11000 E11000 duplicate key error index: cvedb.cves.$_id_  dup key: { : ObjectId('54c7b9d737b9ad76355dcd4b') }

no pytz in requirements.txt

Traceback (most recent call last):
  File "/root/cve-search/sbin/db_mgmt_misp.py", line 16, in <module>
    import pytz
ImportError: No module named 'pytz'

Only Database in Windows - It's possible?

On a computer with Windows 2008 R2 and running MongoDB OK I want to install CVE-SEARCH
 
But I'm only interested in the updated database, I do not want the web interface

I just want to download from Sources (eg http://static.nvd.nist.gov/feeds/xml/cve/) the CVE database and I store them in the local database MongoDB on Windows.

I have installed python3

It's possible?

Error in search_fulltext.py

Hi! have a problem running the command python3 search_fulltext.py -g -s >cve.json

He shows me the following

XXXXX@ubuntu:~/cve-search/bin$ python3 search_fulltext.py  -g -s >cve.json
Traceback (most recent call last):
  File "search_fulltext.py", line 28, in <module>
    ix = index.open_dir("indexdir")
  File "/usr/local/lib/python3.4/dist-packages/whoosh/index.py", line 123, in open_dir
    return FileIndex(storage, schema=schema, indexname=indexname)
  File "/usr/local/lib/python3.4/dist-packages/whoosh/index.py", line 421, in __init__
    TOC.read(self.storage, self.indexname, schema=self._schema)
  File "/usr/local/lib/python3.4/dist-packages/whoosh/index.py", line 619, in read
    % (indexname, storage))
whoosh.index.EmptyIndexError: Index 'MAIN' does not exist in FileStorage('indexdir')

I have Whoosh==2.7.0

Is it a bug? Or am I doing something wrong?

Thank you

AttributeError: 'ExploitHandler' object has no attribute 'cveref'

After setup and in doing an update I get the following error:

Starting d2sec
Traceback (most recent call last):
File "/home/ttrostel/cve-search-master/sbin/db_mgmt_d2sec.py", line 109, in
parser.parse(f)
File "/usr/lib/python3.4/xml/sax/expatreader.py", line 107, in parse
xmlreader.IncrementalParser.parse(self, source)
File "/usr/lib/python3.4/xml/sax/xmlreader.py", line 123, in parse
self.feed(buffer)
File "/usr/lib/python3.4/xml/sax/expatreader.py", line 207, in feed
self._parser.Parse(data, isFinal)
File "../Modules/pyexpat.c", line 459, in EndElement
File "/usr/lib/python3.4/xml/sax/expatreader.py", line 307, in end_element
self._cont_handler.endElement(name)
File "/home/ttrostel/cve-search-master/sbin/db_mgmt_d2sec.py", line 74, in endElement
if self.cveref != "":
AttributeError: 'ExploitHandler' object has no attribute 'cveref'
d2sec has 0 elements (0 update)
Any idea what is wrong or how it can be fixed?

Thanks

db_mgmt_cpe_other_dictionary.py taking a long time

Not sure if this is an issue or it's just the way it works. I'm trying to use the software (great idea btw, thank you for doing this). I'm kind of stuck on the db_mgmt_cpe_other_dictionary.py script.
It starts off well, running its thing quickly. Then after a few hours the process slows way down, according to the progress bar. It's been running for over 12 hours now.
mongostat
mongostat
connected to: 127.0.0.1
insert query update delete getmore command flushes mapped vsize res faults locked db idx miss % qr|qw ar|aw netIn netOut conn time
*0 *0 *0 *0 0 69|0 0 1.03g 2.3g 472m 0 cvedb:0.0% 0 0|0 1|0 7k 7k 3 05:22:20
*0 *0 *0 *0 0 71|0 0 1.03g 2.3g 472m 0 cvedb:0.0% 0 0|0 1|0 7k 7k 3 05:22:21
*0 *0 *0 *0 0 71|0 0 1.03g 2.3g 472m 0 cvedb:0.0% 0 0|0 1|0 7k 7k 3 05:22:22
*0 *0 *0 *0 0 72|0 1 1.03g 2.3g 472m 0 cvedb:0.0% 0 0|0 1|0 7k 8k 3 05:22:23
*0 *0 *0 *0 0 69|0 0 1.03g 2.3g 472m 0 cvedb:0.0% 0 0|0 1|0 7k 7k 3 05:22:24
*0 *0 *0 *0 0 71|0 0 1.03g 2.3g 472m 0 cvedb:0.0% 0 0|0 1|0 7k 7k 3 05:22:25
*0 3 *0 *0 0 69|0 0 1.03g 2.3g 472m 0 cvedb:0.0% 0 0|0 1|0 7k 7k 3 05:22:26
*0 *0 *0 *0 0 71|0 0 1.03g 2.3g 472m 0 cvedb:0.0% 0 0|0 1|0 7k 7k 3 05:22:27
*0 *0 *0 *0 0 71|0 0 1.03g 2.3g 472m 0 cvedb:0.0% 0 0|0 1|0 7k 7k 3 05:22:28
*0 *0 *0 *0 0 71|0 0 1.03g 2.3g 472m 0 cvedb:0.0% 0 0|0 1|0 7k 7k 3 05:22:29

mongotop
connected to: 127.0.0.1

                        ns       total        read       write              2015-05-28T12:23:09
                 cvedb.cpe      1967ms      1967ms         0ms
            cvedb.cpeother         2ms         2ms         0ms
      admin.system.indexes         0ms         0ms         0ms
        admin.system.roles         0ms         0ms         0ms
        admin.system.users         0ms         0ms         0ms
      admin.system.version         0ms         0ms         0ms

                        ns       total        read       write              2015-05-28T12:23:10
                 cvedb.cpe      1917ms      1917ms         0ms
            cvedb.cpeother         1ms         1ms         0ms
      admin.system.indexes         0ms         0ms         0ms
        admin.system.roles         0ms         0ms         0ms
        admin.system.users         0ms         0ms         0ms
      admin.system.version         0ms         0ms         0ms

                        ns       total        read       write              2015-05-28T12:23:11
                 cvedb.cpe      1958ms      1958ms         0ms
            cvedb.cpeother         1ms         1ms         0ms
      admin.system.indexes         0ms         0ms         0ms
        admin.system.roles         0ms         0ms         0ms
        admin.system.users         0ms         0ms         0ms
      admin.system.version         0ms         0ms         0ms

Unix top
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
628 mongodb 20 0 2408028 484064 439436 S 97.1 23.7 1153:57 mongod
1311 root 20 0 985416 794980 7664 S 2.7 39.0 22:46.23 python3

Full Text visualization: Tip/Idea and Question

Hello
It would be possible to add fulltext visualization (search_fulltext.py -g -s) to products and vendors?
I wish make build a list of the most common products and venders, no keywords used in CVE.

They also do not know if it's possible to use -g -s but adding a filter or external command.
Example:

python3 bin/search_fulltext.py  -g -s |  jq 'select ((.Modified >= "2015-01-01" and .Modified <="2015-12-31")) | >cve.json

This I want to make a filter, keywords used in CVE only 2015 year.

Thanks!

Generate UUID for vulnerability without CVE assignment or vendor assignment

As cve-search aggregates various sources of vulnerability, it's not uncommon to get vulnerability without CVE assignment. The idea is to allocate an UUID at the import to ensure a unique identified within
a cve-search installation or beyond cve-search.

The proposal, in a near future, would be to support a public version of cve-search were security researchers could get an UUID for a vulnerability without the need to ask a CVE NA. This UUID could be
used as an unique reference later on even if there is no official CVE assignment or vendor assignment.

Access website from remote location?

Hello

I'm not sure this is the right place to ask about it as it may not be a real issue, but I've trouble with accessing the built-in webserver.

I installed cve-search and in a VM with 2 network adapters. One is configured as NAT, the other is host-only. Apparantly it is not possible to access the VM from the host using port forwarding in NAT: 10.0.2.15:5000 (guest) > 127.0.0.1:5000 (host). Connection via the host-only adapter on 192.168.56.200:5000 doesn't work either.

Is there a setting to specify which IP(s) the built-in server must listens to? I have the impression that it will only accept connections from localhost.

Even with a specific rule to accept connections on port 5000 in iptables, I get the following browser error; "The connection to the server was reset while the page was loading."
Using the host only network's IP, I get "Firefox can't establish a connection to the server at 192.168.10.56:5000."

sudo iptables -A INPUT -p tcp -s 0/0 --sport 1024:65535 -d 192.168.56.200 --dport 5000 -m state --state NEW,ESTABLISHED -j ACCEPT
sudo iptables -A OUTPUT -p tcp -s 192.168.56.200 --sport 5000 -d 0/0 --dport 1024:65535 -m state --state ESTABLISHED -j ACCEPT

Locally on the VM itself, it works only on 127.0.0.1:5000
screenshot from 2015-12-16 09-07-34
Using 192.168.56.200:5000 or 10.0.2.15:5000 it doesn't accept connections either.

Thank you in advance for your time.

API features

Should we integrate all the features from CVE-Search into the API? With this, I mean features like

  • Translating CPEs from 2.2 to 2.3
  • pull vFeed info using CVE-Search
  • etc ...

IRC bot for CVE-Search

I would like to make an IRC bot for CVE-Search. Do I make this within CVE-Search, like the XMPP client, or do I make this a separate project?

pyhton3 db_updater.py error: Import syslog fails on Windows

Update database (pyhton3 db_updater.py) fails on Windows 2008 R2 SP1, Python 3.5.1

pyhton3 db_mgmt.py -p (OK)
pyhton3 db_mgmt_cpe_dictionary.py (OK)
pyhton3 db_updater.py -c (FAIL)

image

C:\cve-search-master\sbin>python db_updater.py -h Traceback (most recent call last): File "db_updater.py", line 17, in <module> import syslog ImportError: No module named 'syslog'

I'm not sure, but ...
Can it be like this?
arista-eosplus/pyeapi#10

CVE-Search more modular

It would be a good idea to make CVE-Search modular and customizable with plug-ins.
Examples of plugins would be:

  • Seen CVEs (current feature)
  • Bookmarked CVEs (current feature)
  • MISP information (current feature)
  • Other information sources (future features)
  • User notes for CVE (requested feature #138 )

Idea: Add "note" in Web Interface.

I do not know if it's possible, but I think it would be good option to add a field to write notes/information/data inside a vulnerability
For example, something like that.

image

It is practical to leave personal comments, for example, a video from youtube, pastebin with script, etc.

Would it be possible?

CPE format

formalizing the CPE format would allow for whitelisting of target software/hardware or other tags, which can be very useful. Example:

(1) WFN:

wfn:[
    part="o",
    vendor="microsoft",
    product="windows_vista",
    version="6\.0",
    update="sp1",
    edition=NA,
    language=NA,
    sw_edition="home_premium",
    target_sw=NA,
    target_hw="x64",
    other=NA
] 

(2) WFN bound to a URI:
cpe:/o:microsoft:windows_vista:6.0:sp1:~-~home_premium~-~x64~-
(3) WFN bound to a formatted string:
cpe:2.3:o:microsoft:windows_vista:6.0:sp1:-:-:home_premium:-:x64:-
(4) Current Format (WFN URI-like notation):
cpe:/o:microsoft:windows_vista:6.0:sp1:~~home_premium~~x64~

In our case, the third option would be better. This way, we can more formal regexes.
The current setup still allows for regex searching (for example by target software or hardware), but is not the nicest. Worth changing?
@adulau @wimremes

Error in dump_last.py

Hi!

I have an error when running the command python3 dump_last.py -r -l 20 -f html

XXX@XXX:~/cve-search/bin$ python3 dump_last.py -r -l 20 -f html
<html><head>
<style>.cve table { border-collapse: collapse; text-align: left; width: 100%; } .cve {font: normal 12px/150% Geneva, Arial, Helvetica, sans-serif; background: #fff; overflow: hidden; border: 1px solid #006699; -webkit-border-radius: 3px; -moz-border-radius: 3px; border-radius: 3px; }.cve table td, .cve table th { padding: 3px 10px; }.cve table tbody td { color: #00496B; border-left: 1px solid #E1EEF4;font-size: 12px;font-weight: normal; }.cve table tbody .alt td { background: #E1EEF4; color: #00496B; }.cve table tbody td:first-child { border-left: none; }.cve table tbody tr:last-child td { border-bottom: none; }.cve table tfoot td div { border-top: 1px solid #006699;background: #E1EEF4;} .cve table tfoot td { padding: 0; font-size: 12px } .cve table tfoot td div{ padding: 0px; }</style>
<title>Last 20 CVE entries</title>
</head><body>
Traceback (most recent call last):
  File "dump_last.py", line 72, in <module>
    if not x['ranking']:
KeyError: 'ranking'

When I add > test.html

XXX@XXX:~/cve-search/bin$ python3 dump_last.py -r -l 20 -f html > test.html
Traceback (most recent call last):
  File "dump_last.py", line 72, in <module>
    if not x['ranking']:
KeyError: 'ranking'

The html generated is blank but watch the source code

<html><head>

<style>.cve table { border-collapse: collapse; text-align: left; width: 100%; } .cve {font: normal 12px/150% Geneva, Arial, Helvetica, sans-serif; background: #fff; overflow: hidden; border: 1px solid #006699; -webkit-border-radius: 3px; -moz-border-radius: 3px; border-radius: 3px; }.cve table td, .cve table th { padding: 3px 10px; }.cve table tbody td { color: #00496B; border-left: 1px solid #E1EEF4;font-size: 12px;font-weight: normal; }.cve table tbody .alt td { background: #E1EEF4; color: #00496B; }.cve table tbody td:first-child { border-left: none; }.cve table tbody tr:last-child td { border-bottom: none; }.cve table tfoot td div { border-top: 1px solid #006699;background: #E1EEF4;} .cve table tfoot td { padding: 0; font-size: 12px } .cve table tfoot td div{ padding: 0px; }</style>

<title>Last 20 CVE entries</title>

</head><body>

If I run the command python3 dump_last.py -r -l 20 -f rss1 > test.xml

I generates XML that contains only this:

<rdf:RDF xmlns="http://purl.org/rss/1.0/" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"><channel rdf:about="http://adulau.github.com/cve-search/"><title>cve-search Last 20 CVE entries generated on 2015-12-17 12:08:46.866382</title><link>http://adulau.github.com/cve-search/</link><description /><items><rdf:Seq /></items></channel></rdf:RDF>

The command I've seen in
"https://github.com/cve-search/cve-search/blob/master/doc/talks/brucon2015-cve-search.pdf"
page 14

Type conversion error on line 159 in "bin/search.py"

Hello,

There's a type conversion error when I'm running "search.py" and outputting the results into a html file.
The issue is on line 159, and the message is "TypeError: Can't convert 'float' object to str implicitly".

I believe the issue can be fixed by changing "item['cvss']" to "str(item['cvss'])".

Cheers.

Problem to use dump or search CVE (CVSS & Time/Date)

I want to create a JSON, XML or CSV file, but only CVSS more 7 and that has been modified between two dates.
Example:

CVSS> 7
Between:
01/11/2015 and 11/31/2015

Or it would be like searching the web but you need the data in JSON / XML or CSV, or export this result in json or others formats.

image

I have tried to do with db_dump.py search.py search_fulltext.py

But I can not find the correct command

Anyone know?

Thank you!

./sbin/db_fulltext.py doesn't create fulltext index

Hello,
maybe this is my specific issue. On Ubuntu 14.04 looks like full text search doesn't work / doesn't generate full text index.
Running command:
./sbin/db_fulltext.py
Processing[##################################################] 5/5
tooks only about 1 second, and the full text querying seems not to work:
./bin/search_fulltext.py -q NFS -q Linux
(...) no output
How to debug / resolve issue?
Thank you!

Cannot run index.py (web interface)

Hi,

I have an issu while running 'python3 index.py' , and it's th same with minimal-web.py

Cmdline Paste :

Traceback (most recent call last):
File "/usr/local/lib/python3.4/dist-packages/tornado/gen.py", line 111, in
from collections.abc import Generator as GeneratorType # py35+
ImportError: cannot import name 'Generator'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "index.py", line 19, in
from tornado.wsgi import WSGIContainer
File "/usr/local/lib/python3.4/dist-packages/tornado/wsgi.py", line 42, in
from tornado import web
File "/usr/local/lib/python3.4/dist-packages/tornado/web.py", line 84, in
from tornado import gen
File "/usr/local/lib/python3.4/dist-packages/tornado/gen.py", line 113, in
from backports_abc import Generator as GeneratorType
ImportError: No module named 'backports_abc'

I have installed tornado, but message still appear.

I also have an error when i tried to install lxml with "pip3 install -r requirement.txt"
Cmdline Paste :

Command /usr/bin/python3 -c "import setuptools, tokenize;file='/tmp/pip-build-42c055cm/lxml/setup.py';exec(compile(getattr(tokenize, 'open', open)(file).read().replace('\r\n', '\n'), file, 'exec'))" install --record /tmp/pip-b5_m1i3k-record/install-record.txt --single-version-externally-managed --compile failed with error code 1 in /tmp/pip-build-42c055cm/lxml
Storing debug log for failure in /root/.pip/pip.log

i run on Debian 8

Could you help me ?

Thanks

Error in index.py ( builtins.TypeError --- TypeError: 'bool' object is not callable)

Hello! I am having a problem (Ubuntu 14.04 64-bit)

I only happens in index.py

If I run minimal-web.py works OK .
But if I run Index.py, then enters 127.0.0.1:5000 shows me the following error

Will I be able to help ? Thank you!

Error in Web Browser

builtins.TypeError

TypeError: 'bool' object is not callable
Traceback (most recent call last)

    File "/usr/local/lib/python3.4/dist-packages/flask/app.py", line 1836, in __call__

    return self.wsgi_app(environ, start_response)

    File "/usr/local/lib/python3.4/dist-packages/flask/app.py", line 1820, in wsgi_app

    response = self.make_response(self.handle_exception(e))

    File "/usr/local/lib/python3.4/dist-packages/flask/app.py", line 1403, in handle_exception

    reraise(exc_type, exc_value, tb)

    File "/usr/local/lib/python3.4/dist-packages/flask/_compat.py", line 33, in reraise

    raise value

    File "/usr/local/lib/python3.4/dist-packages/flask/app.py", line 1817, in wsgi_app

    response = self.full_dispatch_request()

    File "/usr/local/lib/python3.4/dist-packages/flask/app.py", line 1477, in full_dispatch_request

    rv = self.handle_user_exception(e)

    File "/usr/local/lib/python3.4/dist-packages/flask/app.py", line 1381, in handle_user_exception

    reraise(exc_type, exc_value, tb)

    File "/usr/local/lib/python3.4/dist-packages/flask/_compat.py", line 33, in reraise

    raise value

    File "/usr/local/lib/python3.4/dist-packages/flask/app.py", line 1475, in full_dispatch_request

    rv = self.dispatch_request()

    File "/usr/local/lib/python3.4/dist-packages/flask/app.py", line 1461, in dispatch_request

    return self.view_functions[rule.endpoint](**req.view_args)

    File "/home/csirt/cve-search/web/index.py", line 273, in index

     [Display the sourcecode for this frame]  [Open an interactive python shell in this frame] timeTypeSelect, cvssSelect, cvss, rejectedSelect, hideSeen, pageLength, 0)

    File "/home/csirt/cve-search/web/index.py", line 177, in filter_logic

    if current_user.is_authenticated():

    TypeError: 'bool' object is not callable

Error terminal:

127.0.0.1 - - [17/Sep/2015 13:27:26] "GET / HTTP/1.1" 500 -
Traceback (most recent call last):
  File "/usr/local/lib/python3.4/dist-packages/flask/app.py", line 1836, in __call__
    return self.wsgi_app(environ, start_response)
  File "/usr/local/lib/python3.4/dist-packages/flask/app.py", line 1820, in wsgi_app
    response = self.make_response(self.handle_exception(e))
  File "/usr/local/lib/python3.4/dist-packages/flask/app.py", line 1403, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/local/lib/python3.4/dist-packages/flask/_compat.py", line 33, in reraise
    raise value
  File "/usr/local/lib/python3.4/dist-packages/flask/app.py", line 1817, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/local/lib/python3.4/dist-packages/flask/app.py", line 1477, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/usr/local/lib/python3.4/dist-packages/flask/app.py", line 1381, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/local/lib/python3.4/dist-packages/flask/_compat.py", line 33, in reraise
    raise value
  File "/usr/local/lib/python3.4/dist-packages/flask/app.py", line 1475, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/local/lib/python3.4/dist-packages/flask/app.py", line 1461, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/home/csirt/cve-search/web/index.py", line 273, in index
    timeTypeSelect, cvssSelect, cvss, rejectedSelect, hideSeen, pageLength, 0)
  File "/home/csirt/cve-search/web/index.py", line 177, in filter_logic
    if current_user.is_authenticated():
TypeError: 'bool' object is not callable
127.0.0.1 - - [17/Sep/2015 13:27:26] "GET /?__debugger__=yes&cmd=resource&f=style.css HTTP/1.1" 200 -
127.0.0.1 - - [17/Sep/2015 13:27:26] "GET /?__debugger__=yes&cmd=resource&f=jquery.js HTTP/1.1" 200 -
127.0.0.1 - - [17/Sep/2015 13:27:26] "GET /?__debugger__=yes&cmd=resource&f=debugger.js HTTP/1.1" 200 -
127.0.0.1 - - [17/Sep/2015 13:27:26] "GET /?__debugger__=yes&cmd=resource&f=console.png HTTP/1.1" 200 -
127.0.0.1 - - [17/Sep/2015 13:27:26] "GET /?__debugger__=yes&cmd=resource&f=console.png HTTP/1.1" 200 -
127.0.0.1 - - [17/Sep/2015 13:27:26] "GET /?__debugger__=yes&cmd=resource&f=source.png HTTP/1.1" 200 -

Error Screenshot:

Image of Yaktocat
Image of Yaktocat

ZeroDivisionError in ProgressBar

If the length of the collection is zero then there is ZeroDivisionError.
I faced this issue in db_mgmt_cpe_other_dictionary.py when updating
the database.
On line 67 it is checked that list(collections) is not empty.
It seems, however, that the length may be still be 0.
I do not know whether the proper correction is to add testing of
zero length. Anyway, it removed the ZeroDivisionError.

Br,
Raino

Error on first command: No module named 'dateutil'

First of all, my apologies, I'm not a python developer, so I'm really just trying to run the commands from the README to get the cve-search running.

I'm running on a Mac, I think I the right python (python3 -V / Python 3.4.3), and I've run the first command (sudo pip3 install...).

Now from the root of the project directory, I'm running the db_mgmt.py and it's throwing an error (note that if I run without the sbin it just can't load the db_mgmt.py libary):

» python3 ./sbin/db_mgmt.py -p
Traceback (most recent call last):
  File "./sbin/db_mgmt.py", line 23, in <module>
    from lib.Toolkit import toStringFormattedCPE
  File "/Users/remy/Sites/clones/cve-search/sbin/../lib/Toolkit.py", line 11, in <module>
    from dateutil import tz
ImportError: No module named 'dateutil'

Anyone able to help?

For CVEs without a cvss score in NVD cve-search.py reports a bogus cvss score of 5

New CVEs are frequently added in NVD initially without a CVSS score which then gets populated at a later stage, for example have a look at all the last added CVEs. When querying the local cve-search db for such vulnerabilities it will return a bogus CVSS base score of 5.

How to reproduce:

  1. go to https://web.nvd.nist.gov/view/vuln/search
  2. perform a search without any string after marking "Search Last 3 Months" and leaving the rest as default, write down one of those CVEs
  3. open some of the resulting vulnerabilities, the majority should be without a cvss score
  4. perform on your local cve-search installation a db update
    # /cve-search/sbin/db_updater.py
    [...]
  1. query that CVEs you wrote down on step 2
    ~/cve-search/bin$ ./search.py -c CVE-2015-6280
{"Modified": "2015-09-27T22:59:13.090-04:00", "Published": "2015-09-27T22:59:12.013-04:00", "_id": {"$oid": "560874f58864332bfe32fbc6"}, "cvss": 5, "cwe": "Unknown", "id": "CVE-2015-6280", "last-modified": "2015-09-27T22:59:13.090-04:00", "references": ["http://tools.cisco.com/security/center/content/CiscoSecurityAdvisory/cisco-sa-20150923-sshpk/cvrf/cisco-sa-20150923-sshpk_cvrf.xml", "http://tools.cisco.com/security/center/content/CiscoSecurityAdvisory/cisco-sa-20150923-sshpk"], "summary": "The SSHv2 functionality in Cisco IOS 15.2, 15.3, 15.4, and 15.5 and IOS XE 3.6E before 3.6.3E, 3.7E before 3.7.1E, 3.10S before 3.10.6S, 3.11S before 3.11.4S, 3.12S before 3.12.3S, 3.13S before 3.13.3S, and 3.14S before 3.14.1S does not properly implement RSA authentication, which allows remote attackers to obtain login access by leveraging knowledge of a username and the associated public key, aka Bug ID CSCus73013.", "vulnerable_configuration": [], "vulnerable_configuration_cpe_2_2": []}

Note the bogus "cvss": 5. One way to detect such spurious scores in the DB is checking that the components of the CVSS vector are not present.

Adding vulnerabilities where CVE id are not assigned

There are other vulnerability database that we could import into cve-search but those vulnerabilities have often no CVE id assigned (temporary or perpetually) . We should find a clean way to import those and reference back the vulnerabilities (to CPE as an example) even if there is no CVE id.

The idea is to build a new collection where the vulnerabilities without CVE id are imported but referenced with their CPE.

It's an open question. cc/ @PidgeyL

pymongo.errors.InvalidOperation: cannot do an empty bulk insert

Hi,
after ProgressBar correction db_mgmt_cpe_other_dictionary.py
goes forward and on line 86 it tries to insert empty batch to
cpeother which causes the error.

I added following test:

if len(batch) != 0:
    cpeother.insert(batch)
    #update database info after successful program-run
    info.update({'db': 'cpeother'}, {"$set": {'last-modified': icve['last-modified']}}, upsert=True)

which fix this issue.

br,
Raino

CVE search seems to add entries in the past

On several devices already, I noticed that CVE-Search adds entries in the past (several days) without CVSS. I am not sure if this is a CVE-Search issue, an NVD issue, an NVD process update, or just a temporary issue they have, but it would be worth taking a look. Have you noticed something similar, @adulau ?

Enable search text on mongodb version: 3.0.8

Hi all,

I've just installed cve-search. It works like a charm, I can make queries by script...

However I can't enable Fulltext search:

Fulltext search not enabled
The current settings in the database do not allow fulltext search.
If you feel like this should be enabled, please contact your administrator.

However, search text is enable on mongodb by default since version 3.

I tried to apply this command , but the result is the same:

db.adminCommand({"setParameter": 1, "textSearchEnabled":true})

return this:
{ "was" : true, "ok" : 1 }

Could you please, help me?

Regards,

Web interface without admin interface (read-only)

Adding an option to not have any admin interface or write-access to the database on the web interface.

Two updates need to be done:

  • routing in Flask should be updated to enable/disable the URL path for such part
  • template should be updated to not include any reference of the disabled URL paths

What do you think @PidgeyL ?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.