blimmer / cdk-datadog-integration Goto Github PK

Amazon Cloud Development Kit (CDK) logic to integrate your AWS account with Datadog

License: Apache License 2.0

JavaScript 4.52% TypeScript 95.48%

datadog aws aws-cdk aws-cdk-construct datadog-integration aws-integration aws-cloudformation cloudformation hacktoberfest

cdk-datadog-integration's Introduction

AWS Cloud Development Kit (CDK) Datadog Integration

This construct makes it easy to integrate your AWS account with Datadog. It creates nested stacks based on the official Datadog Cloudformation templates using Amazon Cloud Development Kit (CDK).

Warning

⚠️ This construct does not use the newest Datadog CloudFormation template because sensitive parameters must be hard-coded. See DataDog/cloudformation-template#68 for an upstream feature request. ⚠️

This construct will still work, but it cannot be updated to the latest integration template until the upstream issue is fixed. Please add a +1 to DataDog/cloudformation-template#68 to help prioritize it.

Basic Usage

Install the package
```
npm i --save cdk-datadog-integration
```
Or via pypi, NuGet, or GitHub Packages.

Import the stack and pass the required parameters.

import * as cdk from "aws-cdk-lib";
import { MonitoringInfrastructureStack } from "../lib/monitoring-infrastructure-stack";

const app = new cdk.App();
new MonitoringInfrastructureStack(app, "MonitoringInfrastructure");

import * as cdk from "aws-cdk-lib";
import * as secrets from "aws-cdk-lib/aws-secretsmanager";
import { DatadogIntegration } from "cdk-datadog-integration";

export class MonitoringInfrastructureStack extends cdk.Stack {
  constructor(scope: cdk.App, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    const datadog = new DatadogIntegration(this, "Datadog", {
      // Generate an ID here: https://app.datadoghq.com/account/settings#integrations/amazon-web-services
      externalId: "",

      // Create or lookup a `Secret` that contains your Datadog API Key
      // See https://docs.aws.amazon.com/cdk/api/latest/docs/aws-secretsmanager-readme.html for details on Secrets in CDK
      // Get your API key here: https://app.datadoghq.com/account/settings#api
      apiKey: secrets.Secret.fromSecretNameV2(
        this,
        "DatadogApiKey",
        "<your secret name>"
      ),
    });
  }
}

Configuration

Use DatadogIntegrationConfig to set additional configuration parameters. Check out docs for more details on what's available.

Additionally, a CDK Construct is exposed, should you want to add additional customizations vs. using the out-of-the-box Stack.

CDK Version Compatibility

This package is expected to work with all recent versions of CDK v2. It has been tested with 2.1.0 so almost certainly works will all newer versions, and probably works with some older versions too, but is untested.

If you're still on CDK v1, you can use cdk-datadog-integration@1, but this version is unmaintained. Please upgrade to CDKv2.

How it Works

This module uses the CfnStack CDK Construct to import the three CloudFormation stacks referenced by the main Datadog CloudFormation template. By referencing the Datadog-provided templates, you can be confident that the integration works exactly as Datadog intends.

Author

This package is created and maintained by Ben Limmer, a freelance architect and consultant. I love helping businesses of all sizes solve their hardest technology problems. Let's connect if I can be of help!

Contributing

PRs are welcome!

Releasing

To release, merge your PR to main.

cdk-datadog-integration's People

Contributors

Stargazers

Watchers

Forkers

ianmextend frankwese isotoma pistazie hawxy testingiacwithnewaccount hlascelles

cdk-datadog-integration's Issues

Ensure Bucket join works properly

The CloudTrails and LogArchives properties require a comma-separated list of buckets if you're providing more than one. Currently, this repo will join them but, because the CDK representation is a "Token", I'm not sure if it will work properly.

I should validate this. If it doesn't work properly, change the type to accept a single bucket instead of an Array.

CDK core pinned at exactly 1.35.0

See https://github.com/blimmer/cdk-datadog-integration/blob/master/package.json#L48

This gives me problems using this with Typescript, because the version I'm installing is far more recent, so I have two conflicting versions (this might also cause problems without Typescript, but certainly Typescript catches this:

lib/myfile.ts:15:37 - error TS2345: Argument of type 'this' is not assignable to parameter of type 'Construct'.
  Type 'MyStack' is not assignable to type 'Construct'.
    Property 'onValidate' is protected but type 'Construct' is not a class derived from 'Construct'.

I think the answer is to relax the CDK version requirement.

CDK v2 support

Do you have any thoughts on how best to support CDK v2?

I'm not aware of a neat way of doing so while maintaining support for v1. I came up with a solution here https://github.com/isotoma/sops-secretsmanager-cdk, which allowed me to factor out non-CDK code, and provide support for v1 and v2 from one codebase, albeit with quite a lot of very similar, but not quite identical code around the CDK parts, and listing both v1 and be dependencies as peer dependencies.

Do you have any plans about how to achieve this for this project? Major version upgrade and just switch to CDK v2 only? New project for CDK v2? Shared codebase with lots of duplication? Those are the options I can think of.

I might be able to help, but I wanted to decide a direction first.

Set up automated publishing via GitHub actions

This blog post (https://www.matthewbonig.com/2020/01/11/creating-constructs/) explains how to do it.

Upgrade Node

Now that Node 16 is out, we should test against 12.x, 14.x and 16.x

Use semantic-release

That would make publishing a bit easier / more accessible.

Use newer AWS integration CFN templates

Since I last updated this construct, Datadog now uses new CFN templates. The main "quick-start" template is here: https://datadog-cloudformation-template-quickstart.s3.amazonaws.com/aws/main_v2.yaml

I don't think we'll want to use the App Key and, instead, have the user pass their external ID directly.

Add deprecation warning for the Datadog Forwarder

The datadog forwarder is considered deprecated for most new installations. Instead, the serverless extension is recommended: https://docs.datadoghq.com/serverless/installation/nodejs/?tab=datadogcli

Test CDK Output from `index.ts`

We should use the @aws-cdk/assert package to ensure that:

All the Datadog stacks are correctly instantiated
Configuration works properly

Figure out what's wrong with the build

1.2.0 failed to build / release (see https://github.com/blimmer/cdk-datadog-integration/runs/2522519681?check_suite_focus=true). We should try to figure out what went wrong. Is it related to #23? Or is it something else since we haven't released it in a while?

I don't have the time today to look into this, so any help would be appreciated!

Migrate to projen aws-cdk-construct

It would be nice to migrate this project to projen (https://github.com/projen/projen). That way, we could benefit from all the conventions, including officially supported JSII workflows, publishing, etc. It also makes keeping up with best-practices easy (just update projen and re-synth).

This is the paved road for constructs now, so we should use it 😄

jsii.errors.JavaScriptError:
  internal/modules/cjs/loader.js:1033
    throw err;
    ^

  Error: Cannot find module 'source-map-support/register'
  Require stack:
  - /private/var/folders/_j/vkq9ggqd44d0mdfzk5rrfsg1_7f_x5/T/jsii-kernel-R60LIH/node_modules/cdk-datadog-integration/lib/index.js
  - 49
      at Function.Module._resolveFilename (internal/modules/cjs/loader.js:1030:15)
      at Function.Module._load (internal/modules/cjs/loader.js:899:27)
      at Module.require (internal/modules/cjs/loader.js:1090:19)
      at require (internal/modules/cjs/helpers.js:75:18)
      at Object.<anonymous> (/private/var/folders/_j/vkq9ggqd44d0mdfzk5rrfsg1_7f_x5/T/jsii-kernel-R60LIH/node_modules/cdk-datadog-integration/lib/index.js:3:1)
      at Module._compile (internal/modules/cjs/loader.js:1201:30)
      at Object.Module._extensions..js (internal/modules/cjs/loader.js:1221:10)
      at Module.load (internal/modules/cjs/loader.js:1050:32)
      at Module._load (internal/modules/cjs/loader.js:938:14)
      at Kernel._execute (/Users/thomas.poignant/Documents/dev/search-infra/.env/lib/python3.8/site-packages/jsii/_embedded/jsii/jsii-runtime.js:8417:19)
      at Kernel.load (/Users/thomas.poignant/Documents/dev/search-infra/.env/lib/python3.8/site-packages/jsii/_embedded/jsii/jsii-runtime.js:7657:30)
      at KernelHost.processRequest (/Users/thomas.poignant/Documents/dev/search-infra/.env/lib/python3.8/site-packages/jsii/_embedded/jsii/jsii-runtime.js:7446:28)
      at KernelHost.run (/Users/thomas.poignant/Documents/dev/search-infra/.env/lib/python3.8/site-packages/jsii/_embedded/jsii/jsii-runtime.js:7384:14)
      at Immediate._onImmediate (/Users/thomas.poignant/Documents/dev/search-infra/.env/lib/python3.8/site-packages/jsii/_embedded/jsii/jsii-runtime.js:7387:37)
      at processImmediate (internal/timers.js:458:21)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "app.py", line 4, in <module>
    from search_infra.search_stack import SearchInfraStack
  File "/Users/thomas.poignant/Documents/dev/search-infra/search_infra/search_stack.py", line 10, in <module>
    from datadog import Datadog
  File "/Users/thomas.poignant/Documents/dev/search-infra/search_infra/datadog.py", line 5, in <module>
    from cdk_datadog_integration import DatadogIntegrationStack
  File "/Users/thomas.poignant/Documents/dev/search-infra/.env/lib/python3.8/site-packages/cdk_datadog_integration/__init__.py", line 65, in <module>
    from ._jsii import *
  File "/Users/thomas.poignant/Documents/dev/search-infra/.env/lib/python3.8/site-packages/cdk_datadog_integration/_jsii/__init__.py", line 16, in <module>
    __jsii_assembly__ = jsii.JSIIAssembly.load("cdk-datadog-integration", "1.0.1", __name__[0:-6], "[email protected]")
  File "/Users/thomas.poignant/Documents/dev/search-infra/.env/lib/python3.8/site-packages/jsii/_runtime.py", line 40, in load
    _kernel.load(assembly.name, assembly.version, os.fspath(assembly_path))
  File "/Users/thomas.poignant/Documents/dev/search-infra/.env/lib/python3.8/site-packages/jsii/_kernel/__init__.py", line 215, in load
    self.provider.load(LoadRequest(name=name, version=version, tarball=tarball))
  File "/Users/thomas.poignant/Documents/dev/search-infra/.env/lib/python3.8/site-packages/jsii/_kernel/providers/process.py", line 330, in load
    return self._process.send(request, LoadResponse)
  File "/Users/thomas.poignant/Documents/dev/search-infra/.env/lib/python3.8/site-packages/jsii/_kernel/providers/process.py", line 318, in send
    raise JSIIError(resp.error) from JavaScriptError(resp.stack)
jsii.errors.JSIIError: Cannot find module 'source-map-support/register'
Require stack:
- /private/var/folders/_j/vkq9ggqd44d0mdfzk5rrfsg1_7f_x5/T/jsii-kernel-R60LIH/node_modules/cdk-datadog-integration/lib/index.js
- 49

Documentation

Make sure that people know how to use this stack. Ensure documentation exists in code, for ambient types, but also in the README. Maybe there's a way to do this in an auto-generated way?

in-code docs
readme / user docs