The missing API for the CISA Known Exploited Vulnerabilities Catalog. This repository contains the source running at kevin.gtfkd.com
Home Page: https://kevin.gtfkd.com/
👋 Hi! I'm Synfinner.
🌱 I’m currently working on learning more about AI/ML, Golang, Rust, Typescript, and more web development
💬 Happy to chat and learn
⚡ Fun fact, I've dabbled in several spoken/written languages including English (:P), Spanish, Norwegian (Bokmål), Russian, Hebrew, and German
![dependabot[bot] avatar](https://avatars.githubusercontent.com/in/29110?v=4 "dependabot[bot]")
CISA KEV data includes a product field that may be useful for metrics in seeing what products are known exploited. Maybe even a count of the product?
Something like:
/kev?filter=product
JSON represented with something like:
{
exploitedProducts: [
"Windows": 3,
"Solaris and Zettabyte File System (ZFS)": 1
]
}
At present, KEVin is not consistently applying the content type headers. In terms of best practice, we should be responding that the content type is application/json.
At the moment, pulling all KEV entries is being done with a for each vuln. I'd like to use something like Flask's stream_with_context feature.
That way pulling all vulns will stream from MongoDB to the response in a more memory-efficient manner.
I'm tired of the million 404s because of automated requests via browsers looking for these resources.
In it's current form, KEVin has all of it's API resources plastered into that main script.
For the sake of readability and size, I'd like to eventually move all the resources and routes to it's own py file.
During code review, it was found that NVD data structure within documents has changed. Were are investigating a patch to our code so that publication and modified dates for vulns (not KEV entries) are properly handled.
As CISA makes the KEV larger, we're going to need to implement pagination when viewing all vulns. This will have to be done because it will eventually become a performance problem
At present, we are using standard in-memory caching via Flask. I'd like to enhance caching via utilizing Redis. This shouldn't be too much of a lift.
We should be able to implement it via install python redis and then adding a config:
REDIS_IP = os.getenv("REDIS_IP")
cache_config = {
'CACHE_TYPE': 'redis',
'CACHE_DEFAULT_TIMEOUT': 300, # 300 seconds = 5 minutes
'CACHE_REDIS_HOST': REDIS_IP,
'CACHE_REDIS_PORT': 6379, # default Redis port
'CACHE_KEY_PREFIX': 'kevin_' # Prefix for cache keys
}
cache = Cache(app, config=cache_config)
Looking into enhancing query sanitization.
Perhaps doing the following:
#Function for sanitizing input
def sanitize_query(query):
# URL decode the query
query = unquote(query)
# Allow alphanumeric characters, spaces, and common punctuation
query = re.sub(r"[^a-zA-Z0-9\s-]", "", query)
# Trim leading and trailing spaces
query = query.strip()
# Remove extra whitespace from query
query = re.sub(r"\s+", " ", query)
# Encode the query
query = query.encode("utf-8")
return query
At the moment, KEVin's KEV api section allows one to pull KEV entries based on recency.
A feature I'd like to add is sorting based on available baseScore within the NVD data (we're already including this) in mongo documents.
in theory, this could be something like:
class SortBySeverityResource(Resource):
def get(self):
# Get the sorting direction from the URL
sort_direction = request.args.get("sort", "asc")
sanitized_sort_direction = sanitize_query(sort_direction)
# Ensure only valid values are accepted for sorting direction
if sanitized_sort_direction != "asc" and sanitized_sort_direction != "desc":
return {"message": "Invalid sorting direction"}, 400
# Set the default sorting field
sort_field = 'nvdData.0.baseScore' # Field path to baseScore within nvdData array
# Fetch and sort the vulnerabilities
cursor = collection.find()
sorted_vulnerabilities = []
for vulnerability in cursor:
nvd_data = vulnerability.get('nvdData', [])
if nvd_data and nvd_data[0].get('baseScore') is not None:
sorted_vulnerabilities.append(vulnerability)
sorted_vulnerabilities.sort(key=lambda v: v['nvdData'][0]['baseScore'], reverse=(sanitized_sort_direction == "desc"))
sorted_vulnerabilities = [serialize_vulnerability(v) for v in sorted_vulnerabilities]
return sorted_vulnerabilities
This would allow us to have a route such as:
api.add_resource(SortBySeverityResource, "/kev/severity")
From there, users would be able to do something like /kev/severity?sort=asc or /kev/severity?sort=desc
In its current form, KEVin has some mixed concepts of single query parameters and RESTfulness.
An example is the sort mechanism. /kev/severity?sort=desc/asc is not very restful.
A more RESTful approach would be /kev/severity?sort=severity&order=asc
Tracking issue for:
At the moment, builds are failing due to Flask-Caching not supporting Flask 3.
This is being addressed in the Flask-Caching repo: pallets-eco/flask-caching#538
At the moment, KEVin is currently only configured to allow searching for text data within KEV entries.
I'd eventually like to enable searching in the 'all' CVEs collection.
The biggest question will be what fields we should search/index on. This is an example of a ALL CVE document:
{
"_id": "CVE-2023-38831",
"namespaces": {
"cve.org": {
"CVE_data_meta": {
"ASSIGNER": "[email protected]",
"ID": "CVE-2023-38831",
"STATE": "PUBLIC"
},
"affects": {
"vendor": {
"vendor_data": [
{
"product": {
"product_data": [
{
"product_name": "n/a",
"version": {
"version_data": [
{
"version_value": "n/a"
}
]
}
}
]
},
"vendor_name": "n/a"
}
]
}
},
"data_format": "MITRE",
"data_type": "CVE",
"data_version": "4.0",
"description": {
"description_data": [
{
"lang": "eng",
"value": "RARLabs WinRAR before 6.23 allows attackers to execute arbitrary code when a user attempts to view a benign file within a ZIP archive. The issue occurs because a ZIP archive may include a benign file (such as an ordinary .JPG file) and also a folder that has the same name as the benign file, and the contents of the folder (which may include executable content) are processed during an attempt to access only the benign file. This was exploited in the wild in April through August 2023."
}
]
},
"problemtype": {
"problemtype_data": [
{
"description": [
{
"lang": "eng",
"value": "n/a"
}
]
}
]
},
"references": {
"reference_data": [
{
"name": "https://www.group-ib.com/blog/cve-2023-38831-winrar-zero-day/",
"refsource": "MISC",
"url": "https://www.group-ib.com/blog/cve-2023-38831-winrar-zero-day/"
},
{
"name": "https://www.bleepingcomputer.com/news/security/winrar-zero-day-exploited-since-april-to-hack-trading-accounts/",
"refsource": "MISC",
"url": "https://www.bleepingcomputer.com/news/security/winrar-zero-day-exploited-since-april-to-hack-trading-accounts/"
},
{
"name": "https://news.ycombinator.com/item?id=37236100",
"refsource": "MISC",
"url": "https://news.ycombinator.com/item?id=37236100"
}
]
}
},
"nvd.nist.gov": {
"configurations": {
"CVE_data_version": "4.0",
"nodes": [
{
"children": [],
"cpe_match": [
{
"cpe23Uri": "cpe:2.3:a:rarlab:winrar:*:*:*:*:*:*:*:*",
"cpe_name": [],
"versionEndExcluding": "6.23",
"vulnerable": true
}
],
"operator": "OR"
}
]
},
"cve": {
"CVE_data_meta": {
"ASSIGNER": "[email protected]",
"ID": "CVE-2023-38831"
},
"data_format": "MITRE",
"data_type": "CVE",
"data_version": "4.0",
"description": {
"description_data": [
{
"lang": "en",
"value": "RARLabs WinRAR before 6.23 allows attackers to execute arbitrary code when a user attempts to view a benign file within a ZIP archive. The issue occurs because a ZIP archive may include a benign file (such as an ordinary .JPG file) and also a folder that has the same name as the benign file, and the contents of the folder (which may include executable content) are processed during an attempt to access only the benign file. This was exploited in the wild in April through August 2023."
}
]
},
"problemtype": {
"problemtype_data": [
{
"description": [
{
"lang": "en",
"value": "NVD-CWE-noinfo"
}
]
}
]
},
"references": {
"reference_data": [
{
"name": "https://www.group-ib.com/blog/cve-2023-38831-winrar-zero-day/",
"refsource": "MISC",
"tags": [
"Exploit",
"Third Party Advisory"
],
"url": "https://www.group-ib.com/blog/cve-2023-38831-winrar-zero-day/"
},
{
"name": "https://www.bleepingcomputer.com/news/security/winrar-zero-day-exploited-since-april-to-hack-trading-accounts/",
"refsource": "MISC",
"tags": [
"Exploit",
"Third Party Advisory"
],
"url": "https://www.bleepingcomputer.com/news/security/winrar-zero-day-exploited-since-april-to-hack-trading-accounts/"
},
{
"name": "https://news.ycombinator.com/item?id=37236100",
"refsource": "MISC",
"tags": [
"Issue Tracking",
"Third Party Advisory"
],
"url": "https://news.ycombinator.com/item?id=37236100"
}
]
}
},
"impact": {
"baseMetricV3": {
"cvssV3": {
"attackComplexity": "LOW",
"attackVector": "LOCAL",
"availabilityImpact": "HIGH",
"baseScore": 7.8,
"baseSeverity": "HIGH",
"confidentialityImpact": "HIGH",
"integrityImpact": "HIGH",
"privilegesRequired": "NONE",
"scope": "UNCHANGED",
"userInteraction": "REQUIRED",
"vectorString": "CVSS:3.1/AV:L/AC:L/PR:N/UI:R/S:U/C:H/I:H/A:H",
"version": "3.1"
},
"exploitabilityScore": 1.8,
"impactScore": 5.9
}
},
"lastModifiedDate": "2023-08-29T16:02Z",
"publishedDate": "2023-08-23T17:15Z"
}
}
}
For some reason, I built the updater to not use the built-in connection pooling.
During additional code review, we found that there was a logical error in how we were handling data from NIST.
Specifically,
if ("nvdData" not in vulnerability or len(vulnerability["nvdData"]) == 0) or "nvdReferences" not in vulnerability["nvdData"][0]:
The issue lies within that we weren't checking for if there was NOT a vulnStatus within an existing document. Because, surprise, NIST doesn't always have a vuln status published in vulns.
By updating our if statement to
if ("nvdData" not in vulnerability or len(vulnerability["nvdData"]) == 0) or "nvdReferences" not in vulnerability["nvdData"][0] or "vulnStatus" not in vulnerability["nvdData"][0]:
We can check data and update missing records.
At present, KEVin loads ALL of the NVD entries during the update process. This means we are jamming 1000 (as of today) entries into memory and then parsing it.
We should REALLY stream the content, so we don't have to load data all at once.
Add a second filter when requesting a CVE from the KEV so that we can pull JUST references and GitHub PoCs.
Something like:
/kev/CVE-2017-5638?references=exploits
nvdReferences add a nested array containing a tag of 'exploit' and we're also adding GitHub POCs in a githubPocs array.
We can easily create a temp array and create a serializer for just the two data sets.
As most of the documents have a really terrible format, this is mostly going to be a big backend project.
Currently exploring options
CISA recently added a new field to their KEV; knownRansomwareCampaignUse
This should be added whenever possible, as it's good data.
Example entry:
{
"cveID": "CVE-2021-27104",
"vendorProject": "Accellion",
"product": "FTA",
"vulnerabilityName": "Accellion FTA OS Command Injection Vulnerability",
"dateAdded": "2021-11-03",
"shortDescription": "Accellion FTA contains an OS command injection vulnerability exploited via a crafted POST request to various admin endpoints.",
"requiredAction": "Apply updates per vendor instructions.",
"dueDate": "2021-11-17",
"knownRansomwareCampaignUse": "Known",
"notes": ""
}
At the moment, the most up-to-date documentation lives on the static landing page for KEVin.
In reality, there should also be a central place for seeing endpoints and what they do. I.E. Swagger-UI.
At present, KEVin has the key fetching duplicated in the serializers. Same with date functionality.
We can dedupe this via having utility functions.
def serialize_date(date_value, format="%Y-%m-%d"):
if isinstance(date_value, datetime):
return date_value.strftime(format)
return date_value
def extract_keys(data, keys):
return {key: data.get(key, {}) for key in keys}
We could then fetch the data via:
nvd_data = extract_keys(vulnerability.get("namespaces", {}), ["nvd.nist.gov"])
def nvd_seralizer(vulnerability):
nvd_data = extract_keys(vulnerability.get("namespaces", {}), ["nvd.nist.gov"])
return {
'cveID': str(vulnerability["_id"]),
'nvdData': nvd_data["nvd.nist.gov"]
}
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.