arminc / terraform-ecs Goto Github PK

Show how to expose EC2 and container metrics so that they can be picked up by Prometheus

Disclaimer: I don't fully understand the context for this file or this portion of the file, and I'm not especially versed in shell scripting.

These two lines appear to me to have issues.

Should az=$(curl -s http://instance-data/latest/meta-data/placement/availability-zone)
be az=$(curl -s http://169.254.169.254/latest/meta-data/placement/availability-zone)

and should region=$${az:0:$${#az} - 1}
be region=$${az:0:$${az} - 1}

Would you mind briefly confirming for me and possibly explaining the context for this file and these lines?

Show a proper microservices example with. It should include the #10 , #9 , #8 and #6

Show how to push container logs to CloudWatch

I've looked at deploy.sh, I can't seem to find the root of an error I keep getting after cloning master:

./deploy.sh CONTAINER_VERSION=nginx:alpine create

Error parsing parameter 'cli-input-json': Invalid JSON: Expecting value: line 6 column 16 (char 95)
JSON received: {
  "family": "nginx",
  "containerDefinitions": [
    {
      "name": "nginx",
      "image": ,
      "memory": 128,
      "portMappings": [
        {
          "containerPort": 80,
          "protocol": "tcp"
        }
      ]    
    }
  ]
}

Hi, can I use this terraform with a nodeJS app? any documents on how to do so?

I have a suggestion with regard to this challenge of running one task per host on boot.

Another approach some use is to create a service with the deployment constraint distinctInstance to tell ECS to only place one task per instance, then you can either go the easy route and set the service desired count to a really high arbitrary number and forget it, or else use a Lambda function to keep the service desired count in sync with the number of instances in the cluster's autoscaling group. This will cause ECS to always start one of that container on each instance and ECS will restart the tasks as well as if they stop.

Hope this helps!

Instead of running Docker directly create a lambda to be able to nicely run one taks per node (sytem container) trough ECS. https://github.com/miketheman/ecs-host-service-scale

The image used by default does not match the image recommended by AWS for ECS.

This returns amzn2-ami-ecs-gpu-hvm-2.0.20210331-x86_64-ebs:

# Get latest Linux 2 ECS-optimized AMI by Amazon
data "aws_ami" "latest_ecs_ami" {
  most_recent = true

  filter {
    name   = "name"
    values = ["amzn2-ami-ecs-*"]
  }

  filter {
    name   = "virtualization-type"
    values = ["hvm"]
  }

  owners = ["amazon"]
}

The recommended image, mentioned here, can be obtained with:

aws ssm get-parameters --names /aws/service/ecs/optimized-ami/amazon-linux-2/recommended | jq -r '.Parameters[0].Value' | jq .image_id

The image id is amzn2-ami-ecs-hvm-2.0.20210331-x86_64-ebs, and this can be fetched with terraform using:

data "aws_ssm_parameter" "ecs_optimised_ami" {
  name = "/aws/service/ecs/optimized-ami/amazon-linux-2/recommended"
}

resource "aws_instance" "myinstance" {
  ami = jsondecode(data.aws_ssm_parameter.ecs_optimised_ami.value)["image_id"]
}

Since syntax for terraform 0.12 is somewhat different, some changes are necessary for these modules to work. I'll create a pull request if that's OK.

aws_launch_configuration has a dependedcy on iam_instance_profile. Since aws_launch_configuration has lifecycle{ create_before_destroy = true}, all its dependencies and their dependencies should have lifecycle{ create_before_destroy = true} this includes iam_instance_profile and its dependencies.
I would also like to thank you for the example it helped me a lot to get started, sharing is caring <3
(this also implies the chnage of some names to name prefixes)

Show how to add bastion server, maybe we don't need this if we fix #1

Create an EC2 nodes update script to update all nodes without disruption. This is mostly for updates/patches on the OS but also when System containers need to be updated

I believe you will cover this in your "Micro services example" issue, but I'll ask here anyway.

I don't see in the instance you deploy where you specify the ECS service and the docker image for it.

Create an example that shows how to use ELB instead of ALB

On line 7 there is a ] that produces the error

Error: Argument or block definition required

On modules/subnet/outputs.tf line 7: An argument or block definition is
required here.

There are some user_data.sh incompatibilities with Amazon Linux 2 which prevent container instances from booting with connectivity to your ECS cluster.

In modules/alb/main.tf :

resource "aws_alb_listener" "https" {
load_balancer_arn = "${aws_alb.alb.id}"
port = "80"
protocol = "HTTP"

default_action {
target_group_arn = "${aws_alb_target_group.default.id}"
type = "forward"
}
}

Show how to add a database like RDS to the infrastructure

Show a fluentd example on how to push logs to ElasticSearch

I don't understand why that resource is confusingly called https. Please update accordingly.

Show how to make alle the traffic from ALB to microservice to RDS encrypted

Good work on this module.

I had a question. How do you handle autoscaling of ECS hosts?

Don't use SSH but use AWS remote commands to execute commands on the instances. This is more secure and beter audited. http://docs.aws.amazon.com/AmazonECS/latest/developerguide/ec2-run-command.html

Ran Cloudrail against the TF here with basic parameters and found a couple of issues.

1. module.ecs.module.ecs_instances.aws_launch_configuration.launch is not using IMDSv2 (metadata_options). The issue can be important because traffic to the ECS can be direct, and if there's a web app running there, someone could access the metadata service.

2. The log groups are not encrypted (module.ecs.module.ecs_instances.aws_cloudwatch_log_group) and EBS is not encrypted at rest (for module.ecs.module.ecs_instances.aws_launch_configuration). Generally not a biggie, but if used in certain regulated environments, it can be a lot of "fun" to fix this after the stack is deployed.

I can take on a fixing both issues and opening a PR, please let me know if this is of interest.