A Serverless plugin for transporting Cloudwatch log groups within your CloudFormation stack into Elasticsearch.

• Installation
• Usage

Install the plugin in your project:

$ yarn add serverless-es-logs --dev
$ npm install serverless-es-logs --save-dev

Add the plugin to your serverless.yml:

plugins:
  - serverless-es-logs

Define your configuration using the custom configuration option in serverless.yml:

custom:
  esLogs:
    endpoint: some-elasticsearch-endpoint.us-east-1.es.amazonaws.com
    index: some-index

Your logs will now be transported to the specified elasticsearch instance using the provided index.

(Optional) The filter pattern that the Cloudwatch subscription should use for your API Gateway access logs. Default is [event], but you can override this to provide a pattern that will match your custom access logs format. See Cloudwatch filter pattern syntax for more info.

custom:
  esLogs:
    apiGWFilterPattern: '[request_timestamp, apigw_request_id, http_method, resource_path, request_status, response_latency]'

provider:
  logs:
    restApi:
      format: '$context.requestTimeEpoch $context.requestId $context.httpMethod $context.resourcePath $context.status $context.responseLatency'

(Required) The endpoint of the Elasticsearch instance the logs should be transported to.

custom:
  esLogs:
    endpoint: some-elasticsearch-endpoint.us-east-1.es.amazonaws.com

(Optional) The filter pattern that the Cloudwatch subscription should use for your lambda functions. Default is [timestamp=*Z, request_id="*-*", event]. See Cloudwatch filter pattern syntax for more info.

custom:
  esLogs:
    filterPattern: '[timestamp=*Z, request_id="*-*", event]'

(Optional) An option to capture access logs created by API Gateway and transport them to Elasticsearch.

custom:
  esLogs:
    includeApiGWLogs: true

provider:
  name: aws
  logs:
    restApi: true

(Required) The Elasticsearch index that should be applied to the logs.

custom:
  esLogs:
    index: some-index

(Optional) The separator to use when creating the date suffix for the index. Default is ..

The format of the index will be: <index>-YYYY<indexDateSeparator>MM<indexDateSeparator>DD

custom:
  esLogs:
    indexDateSeparator: '-'

This will result in a date like 2020-04-20.

(Optional) The number of days that Cloudwatch logs should persist. Default is to never expire.

custom:
  esLogs:
    retentionInDays: 7

(Optional) Custom tags that should be applied to every log message processed by this plugin and sent to elasticsearch as fields.

custom:
  esLogs:
    tags:
      some_tag: something
      some_other_tag: something_else

(Optional) Override role management for the log processer lambda and use the manually specified default role. Default is false.

custom:
  esLogs:
    useDefaultRole: true

provider:
  name: aws
  role: arn:aws:iam::123456789012:role/MyCustomRole

(Optional) VPC configuration for the log processor lambda to have.

custom:
  esLogs:
    vpc: 
      securityGroupIds:
        - sg-123456789
      subnetIds:
        - subnet-123456789
        - subnet-223456789
        - subnet-323456789

(Optional) Adds AWS Xray writing permissions to the processor lambda. You will need these if you enable tracing for ApiGateway on your service.

custom:
  esLogs:
    xrayTracingPermissions: true

provider:
  tracing:
    apiGateway: true

(Optional) Sets the reservedConcurrency of the lambda

custom:
  esLogs:
    reservedConcurrency: 3

provider:
  tracing:
    apiGateway: true