The ecs-kafka from globallogicpractices

I can't get this running

Hi,

Thanks for sharing this ambitious project. I've been trying to deploy a kafka cluster on ecs using the terraform scripts provided in here, however I've been encountering many issues that I will try to describe below. [The referenced files below are in the terraform/environments/development]

I made sure to have installed the requirements. These are the versions in my system:

Terraform v0.11.8
ansible 2.7.0
Python 2.7.9
boto3
botocore<1.13.0>

Updated secrets.tfvars with the path to my keys
Updated terraform.tfvars with the appropriate values for my use case.
Ran terraform plan -var-file=secrets.tfvars and got

Error: Error loading modules: module vpc: not found, may need to run 'terraform init'

So after running terraform init I get

Initializing modules...
- module.vpc
  Getting source "../../modules/vpc"
- module.glp-private-zone
  Getting source "../../modules/route53-hosted-zone"
- module.bastion
  Getting source "../../modules/bastion"
- module.ecs-kafka-cluster
  Getting source "../../modules/ecs-kafka-zk-cluster"
- module.efs-private-subnet
  Getting source "../../modules/efs"
- module.aws-log-group
  Getting source "../../modules/cloudwatch-log-groups"
- module.ansible-ecs-setup
  Getting source "../../modules/ansible-ecs"

Error: module 'ecs-kafka-cluster': unknown variable referenced: 'kafka_instance_type'; define it with a 'variable' block
Error: module 'ecs-kafka-cluster': unknown variable referenced: 'kafka_asg_max_size'; define it with a 'variable' block
Error: module 'ecs-kafka-cluster': unknown variable referenced: 'efs_kafka_data_dir'; define it with a 'variable' block
Error: module 'ecs-kafka-cluster': unknown variable referenced: 'kafka_asg_min_size'; define it with a 'variable' block
Error: module 'ecs-kafka-cluster': unknown variable referenced: 'kafka_asg_desired_size'; define it with a 'variable' block
Error: module 'glp-private-zone': unknown module referenced: glp-vpc
Error: module "ansible-ecs-setup": "route53_private_domain" is not a valid argument
Error: module "ansible-ecs-setup": missing required argument "log_group_name"
Error: module 'glp-private-zone': reference to undefined module "glp-vpc"

In order to solve the unknown variable referenced errors I added the mentioned variables to the variables.tf file like shown below

// EFS
variable "efs_kafka_data_dir" {}

// ECS Kafka cluster
variable "kafka_asg_max_size" {}
variable "kafka_asg_min_size" {}
variable "kafka_asg_desired_size" {}
variable "kafka_instance_type" {}

Then, in order to solve the Error: module 'glp-private-zone': unknown module referenced: glp-vpc I changed the line 53 in main.tf to use the vpc module instead of using the glp-vpc module.

   vpc_id            = "${module.vpc.vpc_id}"

To get rid of the Error: module "ansible-ecs-setup": "route53_private_domain" is not a valid argument I commented the line 121 in the main.tf file because it says that This is only used for Couchbase server so for now it's ok.

Finally, in order to get rid of the Error: module "ansible-ecs-setup": missing required argument "log_group_name" I added the following line to the module ansible-ecs-setup

   log_group_name                 = "/ecs/${var.environment}-logs"

Only after doing these changes I was able to successfully run terraform init. But then, when I run terraform plan it starts asking for a lot of variables when There should be no manual intervention required.

These are the first three vars it asks for

var.ami_name_regex
  Enter a value: asd

var.ami_owner_name
  Enter a value: asd

var.aws_key_name
  Enter a value:

I find it really hard to run this script. Is there something that I should be doing in a different way?

globallogicpractices / ecs-kafka Goto Github PK

ecs-kafka's People

Contributors

Stargazers

Watchers

Forkers

ecs-kafka's Issues

I can't get this running

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent