Code Monkey home page Code Monkey logo

hids-cloudwatchlogs-elasticsearch-template's Introduction

Monitor Host-Based Intrusion Detection System Alerts on Amazon EC2 Instances

dashboard

Solution Overview

This solution uses Amazon CloudWatch Logs to collect and aggregate alerts in near real time from an open-source security (OSSEC) HIDS. It uses a CloudWatch Logs Subscription to deliver the alerts to Amazon OpenSearch Service for analysis and visualization with OpenSearch Dashboards.

This solution contains a test environment AWS CloudFormation stack that you can automatically provision into an existing Amazon VPC subnet. The CloudFormation template performs the following high-level steps in the region you choose:

  1. Creates two EC2 instances running Amazon Linux with an AWS Identity and Access Management (IAM) role for CloudWatch Logs access. Note: In order to provide sample HIDS alert data, the two EC2 instances are configured automatically to generate simulated HIDS alerts locally.
  2. Installs and configures OSSEC, the CloudWatch Logs agent, and additional packages used for the test environment.
  3. Creates the target HIDS Amazon OpenSearch domain.
  4. Creates the target HIDS CloudWatch Logs group.
  5. Creates the Lambda function and CloudWatch Logs Subscription in order to send HIDS alerts to Amazon OpenSearch.

How the Solution Works

  1. On the target EC2 instances, the OSSEC HIDS generates alerts that the CloudWatch Logs agent captures. The HIDS performs log analysis, integrity checking, Windows registry monitoring, rootkit detection, realtime alerting, and active response. For more information, see Getting started with OSSEC.
  2. The CloudWatch Logs group receives the alerts as events.
  3. A CloudWatch Logs subscription is applied to the target log group to forward the events through AWS Lambda to Amazon OpenSearch.
  4. Amazon OpenSearch loads the logged alert data.
  5. OpenSearch visualizes the alerts in near real time. OpenSearch Dashboards is an open-source visualization tool designed to work with OpenSearch. Amazon OpenSearch Service provides an installation of OpenSearch Dashboards with every OpenSearch Service domain.

diagram

For the following input parameters, you must identify a target VPC and subnet (which requires Internet access) for deployment. If the target subnet uses an Internet gateway, set the AssignPublicIP parameter to true. If the target subnet uses a NAT gateway, you can leave the default setting of AssignPublicIP as false.

You also must provide a trusted source IP address or CIDR block for access to the environment following the creation of the stack and an EC2 key pair to associate with the instances. You can find information about creating an EC2 key pair in the documentation here. Note that the trusted IP address or CIDR block also is used to create the access policy automatically for dashboard access. It is recommended that you utilize a specific IP address or CIDR range rather than using 0.0.0.0/0 which will allow all IPv4 addresses to access your instances.

CloudFormation Parameters

  1. HIDSInstanceSize: EC2 instance size for test server
  2. OpenSearchInstanceSize: OpenSearch instance size In region S3 key for Lambda zipped deployment package
  3. VPCId: An Amazon VPC into which to deploy the solution
  4. SubnetId: A SubnetId with outbound connectivity within the VPC you selected (requires Internet access)
  5. AssignPublicIP: Set to true if your subnet is configured to connect through an Internet gateway; set to false if your subnet is configured to connect through a NAT gateway
  6. MyTrustedNetwork: Your trusted source IP or CIDR block that is used to whitelist access to the EC2 instances and the Amazon OpenSearch endpoint

If you already have a VPC Subnet configured with internet access, you can launch the stack using the CloudFormation Template.

Thanks

Thanks to the OSSEC Project Team for authoring the OSSEC HIDS.

License

https://aws.amazon.com/asl/

hids-cloudwatchlogs-elasticsearch-template's People

Contributors

cgulliver avatar chriscoombs avatar hgs-shoust01 avatar hyandell avatar jpeddicord avatar muhsinoglu avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

hids-cloudwatchlogs-elasticsearch-template's Issues

CloudFormation Error - The runtime parameter of nodejs4.3 is no longer supported for creating or updating AWS Lambda functions.

When creating with CloudFormation I get the following error:

The runtime parameter of nodejs4.3 is no longer supported for creating or updating AWS Lambda functions. We recommend you use the new runtime (nodejs8.10) while creating or updating functions. (Service: AWSLambdaInternal; Status Code: 400; Error Code: InvalidParameterValueException

screenshot from 2019-02-27 12-07-20

I will create a pull request that updates to nodejs8.10.

Issue with input format for Lambda function

I've created a similar solution by referring to this but using OSSEC 2.9.4 where the format of alerts output is changed and the lambda function is unable to process the alerts to Elasticsearch. Error: Process exited before completing request.

Older version format:
{"rule":{"level":6,"comment":"SSH insecure connection attempt (scan).","sidid":5706},"location":"/var/log/secure","full_log":"Aug 6 00:03:57 ip-10-0-0-100 sshd[12462]: Did not receive identification string from 127.0.0.1 port 35578"}

Newer version format:
{"rule":{"level":6,"comment":"SSH insecure connection attempt (scan).","sidid":5706},"srcip":"196.52.43.80","location":"/var/log/auth.log","full_log":"Aug 6 18:49:02 ip-15-0-213-229 sshd[3270]: Did not receive identification string from 196.52.43.80"}

Hoping for a resolution at the earliest possible.

Clarification on OSSEC server and agent

According to the documentation of OSSEC, An OSSEC server will monitor and analyse the log from remote agents. Although it is not clearly clarified in their documentation, a server should be able to monitor itself too.

If my understanding is correct, i don't see remote agents are installed. Instead, two OSSEC servers are installed and the alerts.json are piped to lambda function via cloudwatch log agents.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.