jupiterone / data-model Goto Github PK

Currently, there is a constraint in the base GraphObject schema that all graph object _key values must be at minimum 10 characters. This has lead to some integration developers adding artificial prefixes at the beginning of _key values just to avoid it.

We should consider reducing the constraint to a single character or at least reducing the constraint lower than 10 characters.

cc @aiwilliams @ndowmon @mknoedel

_type should be a snake_case value.

See https://jptrone.slack.com/archives/C016PD96CK0/p1648577338951109?thread_ts=1648228900.094979&cid=C016PD96CK0 for final analysis.

Historically, Firewall Rules have been created as edges between Firewalls and "The Internet". Properties such as Allow/Deny, startIp/endIp, protocol, etc. have been properties on these edges, and to query in J1QL, one must build a query like FIND Firewall THAT (ALLOWS|DENIES) as firewallRule * return firewallRule.*.

Firewall rules aren't always related to Firewalls, though, as is the case with Azure SQL Servers (who have firewall rules with fewer properties, omitting Allows/Denies, Protocol, and other typical properties). I think it would be best to create a FirewallRule entity class.

Related to JupiterOne/sdk#315

Proposal: Namespace global entities under an exported GlobalEntities property.

e.g.

import { GlobalEntities } from '@jupiterone/data-model';

const INTERNET = GlobalEntities.INTERNET;

retentionPeriod should represent the number of days for which the data is retained

Warning when validating against the Host schema:

unknown format "ip" ignored in schema at path "#/properties/ipAddress"

There are types in the integration-sdk coded to reflect the types defined by the schemas in this repository. Likely other projects are doing something similar.

It would be ideal to have a package, born out of this repo, that provides TypeScript type definitions for the schemas of the data model. This project is already shipping as an NPM module, so it is convenient to include the generated types in that package.

ISSUED will represent a relationship between an authoritative entity (e.g. user) and some resource that the authoritative entity issued to another entity (e.g. a token).

Example:

my_user ISSUED the_token

Example: a mathematical or statistical model that is used for machine learning

Examples:

Device -INSTALLED-> Application
user_endpoint -INSTALLED-> Application

See: https://github.com/JupiterOne/data-model/blob/master/src/schemas/Host.json#L14

Example of a case that causes the schema validation to fail:

https://abc.com/<abc>

I think we should consider a more permission type that matches URLs with or without a protocol.

See: https://github.com/JupiterOne/data-model/blob/master/src/schemas/Function.json#L24

Problem

Currently, the Function memorySize property has an inconsistent type between the data model and the AWS integration. In the data-model, the type is listed as a "string" and in the AWS integration, the data type is a "number". Additionally, it's unclear from the name of the property what the memory unit is (e.g. mb, gb, etc.).

Proposal

• Change the data type to a number
• Rename the property to have unit clarity. I think we should standardize on a memory unit size to make it clear in the property name and description fields what the unit is. Google Cloud makes the property name very distinct: availableMemoryMb. Thoughts on updating the property field to match Google Cloud?

The Stethoscope agent and CarbonBlack sensor map to a target entity of _type: 'user_endpoint' while the Snipe-IT integration maps to _type: 'hardware'. Ideally, we'd see a single entity known to Snipe-IT and monitored by those agents.

See: JupiterOne/graph-google-cloud#19 (comment)

Disk should mirror DataStore

These can be used as classes for compliance standards and sections.

This is inconsistent with other classification types in the data model

Example:

{
  "_class": [
    "DataObject"
  ],
  "_key": "my_testing_key",
  "_type": "my_testing_type",
  "name": "John",
  "displayName": "Wick",
  "classification": null
}

Sometimes, the data provider/API of third party systems returns a null value for properties. It doesn't seem that Ajv provides for a global configuration of allowing nullable properties, we'd have to make the schemas have nullable on every single property, or at least on those where we want to allow null.

During work on the PagerDuty integration, it seemed that a better course was to change createIntegrationEntity to avoid transferring null values to the target entity. This seemed at the time as essentially the same behavior as not receiving the property at all from the provider.

There's been another run in where the integration was using assign to directly set null and had to work around validation by value ? value : undefined.

We could consider making https://github.com/JupiterOne/data-model/blob/master/src/validateEntityWithSchema.ts#L7 support an option to remove null value properties from the graph object so that they're treated as undefined.

Ohh, nice find on the @types/json-schema! I wonder if I totally overlooked that or if it's relatively new. I think it would be wise to move to use those types.

Originally posted by @aiwilliams in JupiterOne/sdk#611 (comment)

Example query:

find Database
THAT HAS DataCollection

Example that should work, but throws a validation error:

{
  _type: 'some-type-of-thing',
  _class: 'Control',
  _key: 'some-key-unique',
  name: 'Name of Thing',
  displayName: 'Name of Thing',
  function: ['appsec', 'bug-bounty', 'pen-test']
}