jupiterone / graph-azure Goto Github PK

I'm trying to figure out the best way to add entities/relationships so that we can answer useful questions for azure users.

Given the following goal:
Find all azure principals that have the ability to delete key vaults in the ndowmon-j1dev key vault

Pseudocode:

FIND principal (user, user group, or service principal) \\ could be any of the three
  THAT IS ASSIGNED roleDefinition
  THAT (ALLOWS?) scope (management group, subscription, resource group, or resource) 
  WHERE roleDefinition.notAllows !== Microsoft.KeyVault/vaults/delete and 
        roleDefinition.allows === Microsoft.KeyVault/vaults/delete and
        resource.name === ndowmon-j1dev

FIND (azure_user|azure_user_group|azure_service_principal)
   THAT ASSIGNED azure_role_definition
   THAT ALLOWS (
      azure_keyvault WITH name = key-vault-id OR
      azure_resource_group THAT HAS azure_keyvault WITH name = key-vault-id OR
      azure_subscription THAT HAS azure_resource_group THAT HAS azure_keyvault WITH name = key-vault-id
   )
   WHERE 
      azure_role_definition.notAllows !== (
         * OR 
         */delete OR 
         Microsoft.KeyVault/*/delete OR 
         Microsoft.KeyVault/vaults/delete
      ) AND 
      azure_role_definition.allows === (
         * OR
         */delete OR
         Microsoft.KeyVault/*/delete OR 
         Microsoft.KeyVault/vaults/delete
      )

Questions:

1. Should the services on azure_role_definition.allows become an entity? I think they should just be fields on azure_role_definition but maybe it would be better to say something like THAT ALLOWS Microsoft.KeyVault WITH notAllows !== (* OR */delete OR vaults/delete) AND allows == (* OR */delete OR vaults/delete.
2. Does the J1QL allow for a more elegant parsing syntax so that we wouldn't need to use (* OR */delete ...) for string matching these RBAC permissions?
3. Does J1QL implement any traversal shortcuts for the target scope (keyvault, resource group or subscription)? Something like the * (0 or more) quantifier in regular expressions, e.g. THAT ALLOWS (* THAT HAS)* azure_keyvault WITH name = key-vault-id.
4. Does the relationship ALLOWS make sense for azure_role_definition --ALLOWS--> (azure_subscription|azure_resource_group|<azure_resource (azure_keyvault, etc...)>)

As a comparison, here's a query to look at specific access to an IAM resource:
EDIT: This query is "Which users have access to production data stores via cross-account IAM assume role trusts?"

Find User as user 
   that ASSIGNED aws_iam_role as assignedRole 
   that TRUSTS aws_iam_role as crossAcctRole 
   that ASSIGNED AccessPolicy as policy 
   that ALLOWS as permission DataStore with tag.Production=true as ds 
return 
   user._type, 
   user.displayName, 
   assignedRole.tag.AccountName, 
   assignedRole.displayName, 
   crossAcctRole.tag.AccountName, 
   crossAcctRole.displayName, 
   policy.displayName, 
   permission.displayName, 
   permission.actions, 
   permission.resources, 
   ds.tag.AccountName, 
   ds._type, 
   ds.displayName, 
   ds.classification, 
   ds.encrypted, 
   ds.webLink

An API error occurs at times when requesting transparentDataEncryptions ("The gateway did not receive a response from 'Microsoft.Sql' within the specified time period."), so sometimes we don't have the data for the encrypted property. This then causes the entity in the graph to be bumped on _version, but the raw data is not bumped because the value for encrypted is not found in the database response, but the transparentDataEncryptions response.

We need to avoid changing the encrypted property when we experience transient errors, otherwise alerts and whatnot can be generated with false positives.

API

Alerts on security events that happened on the subscription
Alerts by Resource Group
Recommended tasks that will help improve the security of the subscription proactively
Tasks by Resource Group

Entities

Relationships

For each applicable azure_storage_account entity, ingest additional properties for blob service data protection. We currently ingest blobSoftDeleteEnabled and blobSoftDeleteRetentionDays but tracking and other recovery information should also be ingested:

The Azure integration currently ingests RBAC roles, but there are roles and administrators associated specifically with Azure Active Directory that should be ingested.

In some cases (e.g. personal Azure accounts), the mail field may be null. If the field returns null, it breaks entity validation for the User entity.

Azure also supports otherMails: string[], which is most likely present for azure personal accounts. It's doesn't appear that the target endpoint in DirectoryGraphClient.iterateUsers returns the otherMails property at the moment, unfortunately.

Add soft delete enabled, soft delete retention, and purge protection enabled properties on the azure_keyvault_service entity.

API

All Virtuall Networks
Virtual Networks in Resource Group

Subnets in Virtual Network
Application Security Groups by Resource Group
Interface Endpoins (relation between interface and subnets)

Firewalls in subsciption

Entities

Relationships

The azure integration throws on many steps with

Could not find the resource group "/subscriptions/<subscription-id>/resourceGroups/<resourceGroup-id>" in this subscription.

This causes steps to not complete and resources to not be ingested.

The JupiterOne data model disallows arrays of non-primitive types in entity properties. The endpoints property on azure_storage_account entities is an array that usually contains only strings, but can in some cases contain objects. When this occurs, synchronization fails with

Error: Error uploading collected data (API response: code=ENTITY_UPLOAD_FAILED_VALIDATION, message="1 validation error(s):- Error in entities[0].endpoints (reason=OBJECTS_WITHIN_ARRAYS_NOT_ALLOWED)

If the RoleAssignment.scope is found in the job state, we create a relationship between RoleAssignment and <scope>. However, we're explicitly defining dependsOn steps and targetType metadata to ensure the integration and its documentation captures all target entities that we're currently ingesting.

If a scope is found in the jobState, but _type is not matched, we should alert with a log message that the developer should add dependsOn and _type matcher.

Related to #146

API

Virtual Machines in Recource Group
Disks in the Subscription
Application Gateways in subscription
Load Balancers
Alert Rules - List By Resource Group

Entities

Relationships

Azure Container Instances offers the fastest and simplest way to run a container in Azure, without having to provision any virtual machines and without having to adopt a higher-level service. For more information, see the Azure Container Instances overview.

API

Conainers in Subscription

Entities

Relationships

API

Ingest Azure Kubernetes Service (AKS) Clusters in subscription

Entities

Relationships

This step uses getRawData to get the raw data of an ActivityLogAlertResource. The step does not log if the call to getRawData returns undefined, but this actually represents an issue and should be handled.

Azure VMs can use either "managed" disks or "unmanaged" disks. "Managed" disks are an entity of their own, and this integration properly creates azure_vm|USES|azure_managed_disk relationships.

However, "unmanaged" disks refer to data stored in a storage account's blob container. These relationships are not being created currently, but the entities have the storage container ID, so they should be added.

When upgrading the SDK to v3.3.0, the Azure integration throws a typescript compile error:

Upgrading the SDK to v3.0.0 introduces type checking for relationship _class. The azure integration was previously using the Denies relationship class for certain relationships between security groups.

In order to complete the major version bump, I'm going to remove the DENIES relationships in security groups. However, It'd be great to add the DENIES type to the SDK and add these relationships back in the future.

API

Cosmos Database Accounts in Subscription
Cosmos Databases for Account
Maria DB Servers
Maria DB Database by Server
MySQL DB Servers
MySQL DB Database by Server
[Postgresql Servers]https://docs.microsoft.com/en-us/rest/api/postgresql/servers/list
[Postgresql Databases by Server]https://docs.microsoft.com/en-us/rest/api/postgresql/databases/listbyserver
Redis Caches
Azure SQL Servers
Azure SQL Databases by Server

Entities

Relationships

Each DB has its own Firewall settings which could be ingested as well.

We need to store dates/times on entities as number values. This makes the data consistent, allows for comparisons in queries, etc. See JupiterOne/graph-okta#16 for example.

Understand & handle the following error in storage accounts:

22:04:55.744Z ERROR Local: Step "Storage Accounts" failed to complete due to error. (errorCode="UNEXPECTED_ERROR", errorId="472281fb-6ad1-4607-978c-fea4012372c4", reason="Unexpected error occurred executing integration! Please contact us in Slack or at https://support.jupiterone.io if the problem continues to occur.") (stepId=rm-storage-resources, errorId=472281fb-6ad1-4607-978c-fea4012372c4)
  RestError: Error "SyntaxError: Unexpected token  in JSON at position 0" occurred while parsing the response body - <?xml version="1.0" encoding="utf-8"?>
  <StorageServiceProperties><Cors/><DeleteRetentionPolicy><Enabled>false</Enabled></DeleteRetentionPolicy></StorageServiceProperties>.
      at new RestError (/home/nick/projects/github/graph-azure/node_modules/@azure/storage-blob/node_modules/@azure/core-http/dist/index.js:2359:28)
      at errorHandler (/home/nick/projects/github/graph-azure/node_modules/@azure/storage-blob/node_modules/@azure/core-http/dist/index.js:3288:17)
      at runMicrotasks (<anonymous>)
      at processTicksAndRejections (internal/process/task_queues.js:97:5)

The J1 data model specifies properties that are intended to represent a normalization across all integrations which ingest data classified in the model. For example, all class: "User" entities should have an email, username, etc. This is particularly important for properties such as createdOn (defined for all entities), when a value for that property can be determined. Without this normalization, the query and presentation power of J1 is diminished.

Regarding User entities, it is important to understand that these will be mapped to a Person entity automatically, in order to support finding all user records across data providers which belong to an actual person. It is reasonable to make Person entity properties such as lastName, a good rule for normalizing User properties such as surname (transfer provider surname to User entity property lastName).

Here is the mapping that occurs for other integrations, showing how properties on User entities are transferred to the automatically created Person entities:

{
      "sourceFilter": {
        "_type": [
          "OR",
          ["==", "okta_user"],
          ["==", "google_user"]
        ],
        "employeeType": [
          "AND",
          ["!=", "bot"],
          ["!=", "generic"],
          ["!=", "service"],
          ["!=", "shared"],
          ["!=", "system"]
        ],
        "userType": [
          "AND",
          ["!=", "bot"],
          ["!=", "generic"],
          ["!=", "service"],
          ["!=", "shared"],
          ["!=", "system"]
        ]
      },
      "relationshipProperties": {
        "_class": "IS"
      },
      "relationshipDirection": "FORWARD",
      "targetFilterKeys": [["_class", "email"]],
      "propertyMappings": [
        {
          "sourceProperty": "email",
          "targetProperty": "email",
          "transforms": [
            "toLowerCase"
          ]
        },
        {
          "sourceProperty": ["firstName", "lastName"],
          "targetProperty": "displayName",
          "delimiter": " "
        },
        {
          "sourceProperty": ["firstName", "lastName"],
          "targetProperty": "name",
          "delimiter": " "
        },
        {
          "sourceProperty": "firstName",
          "targetProperty": "firstName"
        },
        {
          "sourceProperty": "lastName",
          "targetProperty": "lastName"
        },
        {
          "targetValue": "Person",
          "targetProperty": "_class"
        },
        {
          "targetValue": "employee",
          "targetProperty": "_type"
        },
        {
          "sourceProperty": "employeeType",
          "targetProperty": "employeeType"
        },
        {
          "sourceProperty": "employeeNumber",
          "targetProperty": "employeeId"
        },
        {
          "sourceProperty": "username",
          "targetProperty": "userIds"
        },
        {
          "sourceProperty": "title",
          "targetProperty": "title"
        },
        {
          "sourceProperty": "manager",
          "targetProperty": "manager"
        },
        {
          "sourceProperty": "managerId",
          "targetProperty": "managerId"
        },
        {
          "sourceProperty": "managerEmail",
          "targetProperty": "managerEmail"
        },
        {
          "sourceProperty": "bitbucketUsername",
          "targetProperty": "bitbucketUsername"
        },
        {
          "sourceProperty": "githubUsername",
          "targetProperty": "githubUsername"
        }
      ]
    },

The Azure integration ingests group members, but does not ingest group owners.

We are currently unable to retrieve Diagnostic Settings for Azure Storage resources (Tables, Queues, Blobs, File Shares). Retrieving Diagnostic Settings for Azure Storage resources is needed to complete CIS Benchmark 3.3, Ensure Storage logging is enabled for Queue service for read, write, and delete requests (Manual).

This benchmark requires that you ensure that the Diagnostic Settings for Storage Queues has a Status that is set to Off and it's Queue Properties include Read, Write, and Delete under the Logging section.

These settings and properties are listed in the classic version of Diagnostic Settings.

We were unable to find an npm package (Azure Client) that allows us to retrieve classic Diagnostic Settings for Azure Storage resources. We were also unable to find a way to create classic Diagnostic Settings using Terraform for Azure resources.

It seems as though the classic Diagnostic Settings are being phased out in favor of new Diagnostic Settings. Azure is calling these settings preview Diagnostic Settings.

These 'preview' settings seem to be the same version of Diagnostic Settings that we were able to successfully retrieve for other Azure resources to complete other CIS Benchmarks, and come from the same Azure Monitor Client that we've been using.

When attempting to use Terraform to create preview Diagnostic Settings for Azure Storage resources, as we've successfully done for other Azure resources, we received the following errors.

This is odd, considering that Azure's documentation says that the newer, preview version of Diagnostic Settings are supported for Azure Storage Resources.

It may be that Terraform does not yet support the creation of preview Diagnostic Settings for Azure Storage resources, even though it supports the creation of Diagnostic Settings for other Azure resources.

Until we are able to find an Azure client that allows us to retrieve classic Diagnostic Settings, or Terraform allows us to create preview Diagnostics Settings for Azure Storage resources, we may not be able to complete or test CIS Benchmark 3.3, Ensure Storage logging is enabled for Queue service for read, write, and delete requests (Manual).

Use Key Vault to safeguard and manage cryptographic keys and secrets used by cloud applications and services.

API

List Vaults in Subscription
Secrets by Vault
Keys by Vault
Certificates by Vault

Entities

Relationships

I have discussed with @aiwilliams that we don't want to generate any placeholder entities for role assignment -> scope targets. If the entity is not found, we should not create a relationship.

The simple implementation creates a problem: what happens if we have a role assignment that points to a known entity type (ex. azure_keyvault_service) that for some reason is not found with jobState.findEntity(keyvaultId)? We want to log that the entity was not found, so developers can see if there is something wrong with the implementation of role_assignment->keyvault_service relationships.

Simply logging any time an entity is not found, however, will create noise for many, many entities that we're not ingesting yet. So we should first check if the resourceId matches a pattern we expect to ingest before looking for that entity. (/subscriptions/<s-id>/resourceGroups/<rg-id>/providers/Microsoft.KeyVault/vaults/<v-id>)

However, if we only create relationships when we explicitly know the matching resource ID pattern, it requires that developers always add matching patterns for new resource entities. If an entity does exist in the job state, but there is not matcher, we should alert developers so they can add the new matching pattern and create relationships between role_assignment and the new entity type.

Since we want to be able to check if a target scope exists in the jobState, the step which generates role-assignment->scope relationships needs to run as late as possible in the dependency tree. This would require a change to the SDK.

Every time a new client is created, a new call to /tokens and /subscriptions endpoints are executed. When new clients are instantiated over and over, this causes a ton of additional unnecessary API calls. We should be able to cache this information for re-use across clients.

APIs:

List CDN Profiles in Subscription
List Endpoints in CDN Porfile
List Origins in CDN Porfile

Entities

Relationships

Ingest information about service principals that can access specific key vaults via the Key Vault Access Policies:

We could add try/catch logic to the iterateAllResources function to allow additional data to be processed in the event of a partial failure. See where the callback is called here: https://github.com/JupiterOne/graph-azure/blob/master/src/azure/resource-manager/client.ts#L351

This would require a way for the integration step to be marked as partial from deep within the client.

A recent release of this project exposed a new error. Examples can be found in cloudwatch:

https://console.aws.amazon.com/cloudwatch/home?region=us-east-1#logsV2:logs-insights$3FqueryDetail$3D$257E$2528end$257E$25272020-09-02T03*3a59*3a59.000Z$257Estart$257E$25272020-09-01T04*3a00*3a00.000Z$257EtimeType$257E$2527ABSOLUTE$257Etz$257E$2527Local$257EeditorString$257E$2527fields*20*40timestamp*2c*20*40message*0a*7c*20filter*20err.stack*20like*20*22Could*20not*20find*20the*20resource*20group*22*0a*7c*20filter*20level*3e40*0a*7c*20sort*20*40timestamp*20desc*0a*7c*20limit*2020$257EisLiveTail$257Efalse$257EqueryId$257E$2527b99beed5-433e-4cd8-89ab-f8dc491a8e6f$257Esource$257E$2528$257E$2527*2faws*2fecs*2fjupiter-integration-azure$2529$2529

The resource group ID needs to be lowercased in order to match properly.

Currently, the azure integration ingests ONLY role definitions that are associated with a role assignment.

Also, add azure_subscription HAS azure_role_definition relationships.

Oftentimes, Azure ARM resource clients don't conform directly with what we have defined as a ListResourcesEndpoint and developers use a pattern of type asserting as below:

     resourceEndpoint: {
        list: async () =>
          serviceClient.eventSubscriptions.listByResource(
            resourceGroupName,
            providerNamespace,
            type,
            name,
          ),
        linkNext: serviceClient.eventSubscriptions.listByResourceNext,
      } as ListResourcesEndpoint,

This can lead to non-compliant interfaces, and the ListResourcesEndpiont type should be updated so that type assertions are no longer required.

When user credentials are invalid, an exception is raise, killing the integration.

{ error_executing: Unexpected error occurred executing integration! We're looking into it.
    at invokeExecutionHandler (webpack:///./src/integration/execution/invokeExecutionHandler.ts?:25:13)
    at process._tickCallback (internal/process/next_tick.js:68:7)
  _cause:
   TypeError: Cannot read property 'map' of undefined
       at Object.fetchAzureData [as default] (./jupiter-integration-azure/src/azure/fetchAzureData.ts:13:12)
       at process._tickCallback (internal/process/next_tick.js:68:7),
  code: undefined,
  statusCode: undefined,
  expose: true,
  correlationId: undefined,
  name: 'error_executing' }

The AzureClient needs some work done to handle non-200 responses.

Ingest Resource Groups from Azure Data service and connect it with users and groups. This data requires to connect things together across Azure services

Resource Groups - List

Roles - List
Roles by Resource Group
Use RBAC to manage access Example

Entities

Relationships

I'm still a fan of just auth, but we'll have to review the world anyway when we work to make things consistent.

Originally posted by @aiwilliams in #229 (comment)

List Openshift Clusters in a subsciption

Entities

Relationships

The azure_sql_server entity has a set of raw data under the encryptionProtector property that may contain information about target key vaults. We should build these relationships if they exist.

Azure App Service used to run web apps, mobile app back ends, and API apps in Azure. Detailed overview.

App Services
Service Cerificates
Certificates

Entities

Relationships

Azure Role Definitions enumerate permissions using four fields:

"permissions": [
  {
    "actions": [
      "*"
    ],
    "notActions": [
      "Microsoft.Authorization/*/Delete",
      "Microsoft.Authorization/*/Write",
      "Microsoft.Authorization/elevateAccess/Action",
      "Microsoft.Blueprint/blueprintAssignments/write",
      "Microsoft.Blueprint/blueprintAssignments/delete"
    ],
    "dataActions": [],
    "notDataActions": []
  }
]

Initially, only blob and file storage account services were enabled in this integration, but a few months back the two remaining services (queue and table) were added.

The integration wrongly logs a message about unhandled storage account services.

The error Invalid client secret is provided should be throwing an IntegrationProviderAuthenticationError for clarity and appropriate handling. However, it throws IntegrationProviderAPIError

Ingest Azure Management Group entities and associated relationships. In particular, RBAC roles should create ASSIGNED relationships where applicable with Azure Management Groups.

Ingest Azure Blueprint entities and any associated relationships.

According to Azure API documentation, Tables and Queues are available in storage accounts of kind "General-purpose V1" (Storage) and "General-purpose V2" (StorageV2). In some cases, API calls to /storageAccounts/<storage-account>/queueServices/default/queues (or /tableServices/default/tables) return 400 errors with FeatureNotSupportedForAccount Queue is not supported for the account.

This has only been seen on storage accounts with kind: 'Storage', but it doesn't necessarily rule out issues with kind: 'StorageV2'.

For each blob storage container in an Azure Storage Account, ingest access policies.

We create direct relationships between azure_role_assignment and principals including azure_user, azure_group, azure_service_principal etc. If the integration instance has the ingestActiveDirectory flag set to false, then it will not be possible to create the required relationships.

If the config value is set to false, we should generate mapped relationships instead of direct relationships.