liatrio / lead-terraform Goto Github PK

View Code? Open in Web Editor NEW

20.0 33.0 17.0 2.35 MB

Terraform modules and scripts to manage LEAD environments

License: MIT License

Ruby 0.04% Makefile 0.69% HCL 65.33% Smarty 23.03% Shell 0.24% Open Policy Agent 1.56% Go 8.16% Mustache 0.96%

lead ignite terraform aws

lead-terraform's Introduction

Overview

This repository contains the Terraform automation to manage LEAD environments.

Additional Documentation

Istio Implementation

Tools

Install required tools with Homebrew:

brew bundle

Setup

You'll need to create a secrets/ directory with a file for each environment (e.g. liatrio-sandbox.tfvars) containing the secrets for that account.

Required Terraform variables: See Slack Operator to setup Slack App.

artifactory_license          = "Artifactory License (ask in slack)"
slack_bot_token              = "Slack Bot User OAuth Token"
slack_client_signing_secret  = "Slack App Credentials Signing Secret"
keycloak_admin_password      = "Keycloak Admin Password"
prometheus_slack_channel     = "Some Slack Channel"
prometheus_slack_webhook_url = "Some Slack Webhook Url"

See instructions for creating slack app

Testing

The tests folder contains functional test which apply individual Terraform modules and verify the final state. The tests use Terratest which uses golang tests to trigger Terraform and verify the outcome. The tests can be run with a local Kubernetes cluster (docker-for-desktop, minikube, microk8s, etc) or create an EKS cluster and run the tests there.

Local tests

Make sure your current Kubernetes context points to your local cluster

make test

AWS tests

The AWS tests create an EKS cluster, run the tests against the cluster and teardown the cluster. This usually takes 25 to 30 minutes.

The tests will not interfere with other clusters in the same account and multiple tests can safely run at the same time. You should run the tests in the sandbox account and you must use a role with sufficient privileges (administrator).

aws-vault exec AWS_PROFILE -- make test-aws

The tests will attempt to teardown the cluster on failure but sometimes it is necessary to manually delete the EKS cluster and VPC.

To speed up running tests repeatedly the --destroyCluster flag can be set to false to skip tearing down the cluster and re-use it on subsequent tests. The test-aws-nodestroy make target uses this flag.

aws-vault exec AWS_PROFILE -- make test-aws-nodestroy

Don't forget to run the test with --destroyCluster set to true to cleanup the cluster.

aws-vault exec AWS_PROFILE -- make test-aws

Running locally

Follow Setup instructions above and store secrets in secrets/docker-for-desktop.tfvars

Add these additional dependency versions in local/environment/local.auto.tfvars

sdm_version                  = "v2.0.0"
dashboard_version            = "v2.0.0"
builder_images_version       = "v2.0.0"
jenkins_image_version        = "v2.0.0"

You can find up to date versions in the lead-environments repo by looking at what is deployed to production or sandbox environments.

To test an environment locally, run:

# Setup keycloak plugin
make plugins

cd local/environment

# Validate a 'stack'
terragrunt validate

# Apply a 'stack' 
terragrunt apply

To test the product locally, run:

cd local/product

# Apply a 'stack' 
terragrunt apply

lead-terraform's People

Contributors

Stargazers

Watchers

Forkers

sadym johnknapprs mrparkers cplee gaip visualizeit-consulting danek2003 eric-info-li nsaishiva damindap syllogy jemalcloud calsaviour gitops-org scotts-byte 13archit ixolt

lead-terraform's Issues

Create `apps` Terragrunt config under `environments/azure`

There should now be example Terraform in the stages/apps directory. We now need to create the Terragrunt configuration to be used in the app stage that will apply the Terraform in stages/apps. During this stage, the Terragrunt should utilize outputs from the cloud-provider Terraform.

apps Terragrunt configuration in the temporary environments directory.
Should run after cloud-provider Terragrunt.
Utilize a resource created in the cloud-provider via outputs.

Add output for random_password

Output updated for provider "random"

Improve Homebrew OS Compatiblity

Homebrew has merged with the Linuxbrew project.

Improve the Brewfile to make it easier to Linux users to use this project's dependency requirements.

Create environment structure for Azure

Create directory structure and supporting files for azurerm

Update Makefile to include Azure processes

Since we are adding an Azure Environment to the list we need to make sure that it works with the existing build/deployment automation. There currently exists a Makefile in the top-level directory that needs to be updated. Make sure the following is being used:

update validate step with Azure environment information
Create test-azure step with temporary message

Move temporary Terraform code in `stacks` directory into correct location in `stages` directory

Our Terraform Azure code currently lives in stacks but we're going to a "stage" approach. Update the repository to reflect this change.

AC:

Move Terraform code for Azure VM into stages/cloud-provider
Terragrunt updated to reflect change

Create temporary Azure Resource via Terraform in `stages/apps` directory

We need a second stage to be run after the cloud-provider stage. In the desired state, the app stage will run the Terraform found in stages/apps/lead directory. The apps/ Terraform will be applied to the cluster resources created during the cloud-provider stage.

Note: Terragrunt configuration for the apps stage will be completed in a later issue

AC:

Any Azure resource to be created
Use resource created in the Azure cloud-provider (i.e. vnet, subnet, etc)
Store Terraform code in a sibling directory to stages/apps/lead

Replace temporary Azure VM with an AKS cluster

Now that we understand the stage process with Terragrunt we are in the position to replace the example vm with the beginning of an AKS cluster.

Remove Azure VM from cloud-provider Terraform
Create Basic AKS cluster in cloud-provider Terraform

Initial Terragrunt configuration

Create Initial Terragrunt configuration

Point to terraform source in stacks/environment-azure
Set up remote backend information

Create dummy resource to be used to verify terragrunt/terraform working in Azure

Once the Terragrunt backend is created and being used we should be able to create resources in Azure. We will eventually be creating using AKS but until then, let's try to spin up another resource to verify our Terragrunt workflow and implementation

Create a resource (Resource Group, VNET, VM, etc)
Use Terragrunt backend from #367

Create Backend in Azure using Terragrunt

We need to create an Azure Backend with Terragrunt. This will be eventually moved over to a different location.

We will need to create some directory structure (environments/azure) for this work to live in.

Create an environments directory on the top level
Create an azure directory in environments
Create an apps directory in azure
Create an azure directory in azure yes... seriously
Create a config directory in azure
Create a terragrunt.hcl file in environments/azure/azure. This will likely need secrets/information to be obscured. Conversation needed.

Note: may need .gitkeep files in apps and config

Move intial azure setup into master

Removing intial_azure_terraform as a long lived branch