Guidance is needed on the folder structure and location for terraform modules, scripts, and other artifacts used to deploy the solution.

Description

As a developer, I want to run integration tests for each CI build run
Running end to end integration tests to validate terraform deployments are operating as expected is essential for enterprises. Similar to bedrock, we'll use terratest in the spirit of following azure devops guidance for terraform deployments. Initially, we'll run a light weight E2E test similar to bedrock's azure-simple integration test.

Creating the terraform template is out of scope for this feature. We'll use the app service terraform hello world sample here as our temporary test bed placeholder until the azure-simple template is released. This integration test should run terraform init, plan, format, destroy and apply on the mocked release and assert that it can establish a connection to the provisioned app service website with a 200 response.

This azure devops build step will get triggered for pull request updates/submissions. The integration test harness should support both local and cloud runs. To simplify local dependency setup we'll use the docker base image from feature #55. We'll need to add a docker build step to the pipeline as the integration test(s) will run as a container.

Acceptance Criteria

• Should assert that we have a mocked terraform template which our int test can apply which scaffolds a hello world app service website.
• Should assert that we have a go terratest module that acts as an integration test which validates the response status code of the provisioned app service website.
• Should assert that the go test run's within a docker container using mage and runs successfully locally and in azure devops.
• Should assert that the README in the test directory is updated with local and cloud setup instructions.

Also, here are a few points that need to be addressed:

1. Issues #55 and #54 are pre-req's

Resources

Mockups
Include an animated GIF of the build running successfully in azDO

Tasks

*Add the go test harness module to the test folder
*Add a mocked app service website terraform template within environments. This is a temporary solution to validate the test module until azure-simple is released.
*Update the documentation in the test folder.
*Dockerize the test harness run

Tasks:

Configure TM with a 10 sec polling timeout
Configure TM to retry twice
Configure TM to poll each location every 20 sec

Application-specific Azure resources are the resources used to host an application and have a different management life-cycle than shared resources. Application-specific resources include the following:

• CDN (Akamai)
• App Service Plan
• Azure App Service (Web App for containers)
• Traffic Manager
• Application Gateway
• Cosmos Database
• Application Insights
• Azure Monitor

Acceptance Criteria:

• Resources are provisioned in a single resource group.
• Terraform is used to describe the infrastructure.
• Ability to describe existing shared resources and integrate the application specific resources with existing shared resources.
• User can run a script from the command-line to deploy the template.

Tasks:

• Documentation
• ARM Azure tool to test the site performance.

Description

As a developer, I want to setup a docker image base for terraform unit and integration testing
Terraform unit and integration test modules will be written in GoLang to remain consistent with Azure infra best practices and bedrock tests. The goal of this feature is automating the pre-requisite installation steps and provide a base docker image that installs:

• Go programming language: Terraform test cases are written in Go.
• dep: dep is a dependency management tool for Go.
• Azure CLI
• mage: simplifies running Terratest cases.

Acceptance Criteria

• Should assert that I can build the image successfully on both a windows and mac machine.
• Should assert that go, terraform, azure cli, dep and mage are installed when ssh'd into the container docker run -it $(IMAGE_NAME) /bin/sh
• Should assert that the image is pushed to a public docker registry
• Pair with @NathanielRose and verify that this image will work for bedrock unit and integration tests.
• Should add the Dockerfile to the root of the test directory
• Should assert that the golang version is a docker build argument.
• Should update the README in the tests directory on sample docker usage.

Resources

Mockups
Screenshots of the golang, terraform, azure cli and mage versions printed out when ssh'd into the docker container.

Tasks

• Add a baseline Dockerfile as per the above story details.
• Push docker image to a public / central docker registry.
• Update README in tests

The requirement here was related to allowing connection via ExpressRoute back to their on-prem data center. This was so that the app can connect to services running on-prem.

This is about connecting to Contoso's private network, not about providing a private service endpoint.

My only knowledge of how to do this is that I found this Azure article about how to do it:

https://docs.microsoft.com/en-us/azure/app-service/web-sites-integrate-with-vnet

You'll see that the solution is explicitly NOT for ASE, as you get this "for free" with ASS. have heard John Doe and others assert that such a thing can't be done, but this article contradicts that assertion.

Notes:

• I don't know if this solution works.
• If it does work, I don't know if you can peer and route to an ER vnet.
• If is does work with ER, I don't know if Contoso would approve the approach.
• John Doe has said that ExpressRoute connectivity can be left out of scope for now. I suspect he's saying that because he believes it can't be done currently.
• We can probably consider this lower priority for now, but it is absolutely a longer term goal to enable App Service-to-ExpressRoute functionality.

Tasks:

• Deploy separate Green-Blue environments
• Configure auto-scaling to dynamically resize green and blue environments as traffic increases/decreased

Description

As a developer, I want to auto trigger a release to QA for merged pull requests
We want to deploy a source terraform build artifact to an azure QA environment when pull requests are merged to master. This will be represented as an azure devops release pipeline. The source artifact will comprise of the terraform template to deploy. The artifact deployed to QA should match what was tested and deployed to the integration environment. We should also run the integration tests after the artifact is applied in the QA env.

Acceptance Criteria

• Should assert that there's a gated release candidate created targeting the QA stage following a PR merge.
• Should assert that the integration tests pass once the release candidate is approved.
• Should assert that you can manually access the app service website resource in QA.
• Should assert that the README in the azure devops folder provides step-by-step instructions with setting up the release pipeline in azure devops including animated gifs.

Resources

Mockups
Include screenshots of a successful release in QA from azure devops.

Tasks

*Update the readme documentation in the azure devops folder.
*Setup release pipeline in azure devops
*Setup variable group sourced by keyvault to resolve secret variables like service principal credentials.

Description

As an engineer, I would like to automate the testing of my infrastructure deployment for a pull request, so that reviewers do not need to manually run the automated tests locally

Acceptance Criteria

scenario 1: new pull request triggers test run

when a new pull request is created
then a ci build that executes the automated tests should be triggered
and the ci build should use the pr branch (/refs/pull/99/merge)

scenario 2: new commits on existing pull request triggers test run

given a pull request exists
when new commits are pushed to the pull request source branch
then a ci build check that executes the automated tests should be triggered
and the ci build should use the pr branch (/refs/pull/99/merge)

scenario 4: ci build in progress blocks pull request merge

when a ci build has been triggered for a pull request
then the pull request merge is blocked

scenario 5: ci build failure blocks pull request merge

when the ci build fails
then the pull request merge is blocked

scenario 6: ci build completes and tests pass

given all other pull request conditions have been met (cla check, reviewers approved)
when the ci build completes successfully
then the pull request can be merged to master

Description

Shared Azure resources are resources or services that are managed by a different group/team and have a different management life-cycle than application-specific resources or services.

This pattern assumes the following shared Azure resources and services:

• Azure Key Vault
• Virtual Network

Requirements

• Azure CLI
• Terraform
• Key Vault
• Azure AD (AAD)

Acceptance Criteria:

• A resource group is provisioned.
• A Key Vault resource is provisioned in the resource group.
• User can run a script from the command-line to deploy the template.

Given input company 'cblt'
and input location 'eastus'
when terraform apply is run
then a resource group in East US exists with the name 'core-infra-eus-cblt'
and a Key Vault exists with the name 'core-kv-eus-cblt' in resource group 'core-infra-eus-cblt'

Notes

The resource group created in #30 can be repurposed for the resource group needed here.

Description

As an engineer, I would like to be able to validate my core environment templates, so that I prevent faulty template changes from being deployed

Acceptance Criteria

Scenario: Changes merged to master have valid syntax
given code changes to the core environment templates have valid syntax
when a pull request is opened for these changes
then a build should be triggered
and the build should pass

Scenario: Changes merged to master have invalid syntax

given code changes to the core environment templates have invalid syntax
when a pull request is opened for these changes
then a build should be triggered
and the build should fail
and the build log should indicate the validation errors that exist in the templates
and the pull request merge should be blocked

Notes:

• Clarify "syntax" language.
• Make sure we're clear that this is a master branch validation.

Acceptance Criteria:

• Domain Name: entref-appservice-containers.net
• SSL wildcard certificate for domain name.
• DNS records updated to route to Traffic Manager.

Tagging @timfpark

Description (Story Incomplete)

As a developer, I can configure and deploy Azure Application Gateway

For Cobalt, Application Gateway should provide:

• Standard DDos protection
• Web App Firewall (cross-site scripting, SQL injection, etc.)
• Reverse proxy API and app traffic

Acceptance Criteria

Reference: [Done-Done Checklist] (https://github.com/Microsoft/code-with-engineering-playbook/blob/master/Engineering/BestPractices/DoneDone.md)

• No hardcoded variables in module config; everything set through Terraform template
• Usage referenced in Azure Simple template
• Screenshot showing resource provisioned in Azure Portal

Also, here are a few points that need to be addressed:

1. No hardcoded variables in module config; everything set through Terraform template

Resources

Tasks

Stories are intended to be completed in a single sprint; if task breakdown creates addition work then team should discuss promoting the Story to an Epic.
Reference: [Minimal Valuable Slices] (https://github.com/Microsoft/code-with-engineering-playbook/blob/master/Engineering/BestPractices/MinimalSlices.md)

Reference: [How to Write Better Tasks] (http://agilebutpragmatic.blogspot.com/2012/04/splitting-story-into-tasks-how-to-write.html)

Assignee should break down work into tasks here

Description

As a developer, I want a AzDO build pipeline triggered for all PR requests/commits.

This feature is intended to add a new azure devops build pipeline that's linked to a definition yml file from the /devops/providers/templates/azure-pipelines.yml in the cobalt repo. The new build definition should be created in the cobalt azure devops instance here. This build pipeline should be triggered from newly submitted PR's and/or new commits to existing PR's.

We should also add the necessary folder filters to avoid running builds for documentation changes.

Acceptance Criteria

• Should assert that the checked in azure devops pipeline definition successfully passes in the Azure DevOps tool for newly posted PR's and new PR commits.
• Should assert that the build definition is running on an Ubuntu VM Image.
• Should include a screenshot of the passed build.
• Should assert there's a mocked inline script step with a sample echo message. This will be replaced once the user story for adding the unit test step is complete.

Technical Design Document

• Add a CI devops flow diagram scoped to the integration environment within the README.md
• Fill in the README in the azure-devops folder to describe the intent / usage of the build pipeline. We should also include step-by-step instructions on how to setup a new Azure DevOPS build pipeline as well as the GitHub repo connection.
• Add the azure devops build status badge to the README in the root.

Also, here are a few points that need to be addressed:

1. This feature is dependent on a azure devops service connection linked to the github cobalt repo.

Resources

Please include a screen shot

Tasks

• Add azure devops pipeline definition
• Add/update README documentation in both the root and azure-devops folders.

Description

As a developer, I have a Provider that configures Azure infrastructure for Terraform
This should set up the basic Azure infrastructure to run Terraform.

Acceptance Criteria

Reference: [Done-Done Checklist] (https://github.com/Microsoft/code-with-engineering-playbook/blob/master/Engineering/BestPractices/DoneDone.md)

• Running on the correct Azure subscription
• Authenticates correctly
• …

Resources

https://www.terraform.io/docs/providers/azurerm/index.html

Tasks

Stories are intended to be completed in a single sprint; if task breakdown creates addition work then team should discuss promoting the Story to an Epic.
Reference: [Minimal Valuable Slices] (https://github.com/Microsoft/code-with-engineering-playbook/blob/master/Engineering/BestPractices/MinimalSlices.md)

Reference: [How to Write Better Tasks] (http://agilebutpragmatic.blogspot.com/2012/04/splitting-story-into-tasks-how-to-write.html)

Assignee should break down work into tasks here

Description

As an engineer, I would like to track what infrastructure is deployed to an environment from a shared/remote backend, so that the deployment changes can be determined from any machine.

Acceptance Criteria

scenario 1: deployment with no existing resources

given a deployment is defined to deploy the following resources

• resource group 'core-use-rg-dev-cblt'
• vnet 'core-use-rg-dev-cblt' in resource group 'core-use-rg-dev-cblt'
• key vault 'core-use-kv-dev-cblt' in resource group 'core-use-rg-dev-cblt'

and none of the above resources exist
when the deployment is executed
then the following resources are created

• resource group 'core-use-rg-dev-cblt'
• vnet 'core-use-rg-dev-cblt' in resource group 'core-use-rg-dev-cblt'
• key vault 'core-use-kv-dev-cblt' in resource group 'core-use-rg-dev-cblt'

scenario 2: deployment with some existing resources

given a deployment is defined to deploy the following resources

• resource group 'core-use-rg-dev-cblt'
• vnet 'core-use-rg-dev-cblt' in resource group 'core-use-rg-dev-cblt'
• key vault 'core-use-kv-dev-cblt' in resource group 'core-use-rg-dev-cblt'

and the following resources exist and were deployed from machine a

• resource group 'core-use-rg-dev-cblt'
• vnet 'core-use-rg-dev-cblt' in resource group 'core-use-rg-dev-cblt'

when the deployment is executed machine b
then the following resources are created

• key vault 'core-use-kv-dev-cblt' in resource group 'core-use-rg-dev-cblt'

scenario 3: deployment with all resources existing

given a deployment is defined to deploy the following resources

• resource group 'core-use-rg-dev-cblt'
• vnet 'core-use-rg-dev-cblt' in resource group 'core-use-rg-dev-cblt'
• key vault 'core-use-kv-dev-cblt' in resource group 'core-use-rg-dev-cblt'

and the following resources exist and were deployed from machine a
when the deployment is executed from machine b
then no new resources are created

Notes

Use multiple directories (clone) to simulate executing the templates from different machines.

Out of scope

Automating the creation of the storage account (remote backend)

• Need to determine if DDoS service is required in addition to App Gateway
• This requires a test to validate that it works

Acceptance Criteria:

• Application gateway is configured with end-to-end SSL. Details here.
  • Application gateway is configured for protocol TLSV1_2.
  • Application gateway is configured to disable TLSV1_0 and TLSV1_1.
• SSL Certificate is stored in Key Vault.
• App Service (Web App) is configured for SSL.
  • Protocol: HTTPS only, minimumTLSV1_2, no incoming client certificates
  • SSL Binding: SNI-SSL
• Traffic Manager profile is configured to monitor endpoints over HTTPS / port 443 only.

Description

The ./setup/acr-sp-init.sh script does not conform to the naming convention standard for this project.

Required changes

• Add parameter ( -a ) for application name.
• Add parameter ( -s ) for suffix.
• Remove parameter ( -p ) for company name.
• Update instructions in README.

Acceptance Criteria

Given script input parameter appname 'CbltApp'
and script input parameter location 'eastUS'
and script input parameter suffix 'Cntso Dev'
when the script is executed
then a resource group exists in East US with the name 'cblt-usea-rg-cntsodev',
and an ACR exists with the name 'cblt-usea-acr-cntsodev',
and an AAD service principal exists with the name 'cblt-usea-adsp-cntsodev-pull', scoped to the ACR resource, and role set to 'acrpull',
and an AAD service principal exists with the name 'cblt-usea-adsp-cndtsodev-push', scoped to the ACR resource, and role set to 'acrpush'.

Artifacts from last merge.

Merge markers on top and bottom of document.

Content repeats at line 115

Story

As an engineer, I would like to automate the testing of my infrastructure deployment from local command, so that a reviewer can quickly test a pull request

Acceptance Criteria

Scenario 1: all tests passes

given the infrastructure deployed is what is expected
when i execute automated tests from command line
then a success message should be returned indicating the test run passed
and the command line exit code should be zero (so command line recognizes as successful)

Scenario 2: one or more tests fail

given the infrastructure deployed is not what is expected
when i execute automated tests from command line
then an error should be returned indicating the test run failed
and the tests that failed should be identified
and the command line exit code should not be zero (so command line recognizes as failure)

Documentation that explains the pattern for writing automated tests. For example, file structure/organization, any resuable patterns to create new tests. This should guide new engineers how to write the automated tests.

Acceptance Criteria

• include an architecture diagram,
• include a narrative about the architecture diagram, how resources relate to one another, and some guidance on best practices in applying the reference architecture,
• compare and contrast to an AKS architecture,
• include guidance on how to deploy the infrastructure.

Some details pertaining to the infrastructure can be obtained from here.

Description

As an IT admin or developer, I need to provision a virtual network as a resource in the shared/core services template so that I can provision additional resources that require an existing virtual network.

Acceptance Criteria

Given input app with value cblt
and input location with value eastus
and input suffix with value core
when terraform apply is run
then a resource group cblt-usea-rg-core should exist in East US
and contain a virtual network cblt-usea-vnet-core
and contain a key vault cblt-usea-vnet-core.

Requirements

• Can be executed at a command prompt.
• Can be integrated into CI/CD.
• Automated test needs to be updated.

Notes

• A subnet will be required. Check on the requirements for App Gateway to see what the name and/or CIDR notation should be for that subnet.

Description

Intentionally lumping three things into this as they are all dependent on one another and each are relatively small in their effort. This will be focused on setting up the dev environment for the terraform templates. It will also establish the readme docs that explains how to execute the templates as a dev from vs code and as a release engineer from command prompt.

TODO: inputs and resource group name may change. Rick trying to determine if there is a way to use a hash function for uniqueness in terraform. ARM supports this but terraform does not. This example uses an alternative method for uniqueness but results in longer names.

Stories

• As an development engineer, I would like my terraform inputs to be different unique, so that my names do not overlap with other engineers
• As an development engineer, I do not want my terraform inputs to be checked into source control, so that other engineers do not modify my infrastructure
• As a release engineer, I would like to be able to deploy terraform templates using only a command line, so that I do not need to setup development tooling to deploy the templates

Acceptance Criteria

• Development Engineers can define terraform inputs so that they do not have to be provided each time they run terraform validate, plan, or apply
• Development Engineers can define terraform inputs in isolation and those inputs are not checked into source control
• Release Engineers can deploy the terraform templates using only terraform and command line. Variables/inputs needed for this case should be documented within the readme.
• Resource group is deployed to demonstrate template does create resources in azure (see given/when/then scenario below)

given input app name 'cblt'
and input env 'dev'
and input location 'eastus'
and input org 'cse'
when terraform apply is run
then a resource group 'rg-cblt-dev-eus-cse' exists

Requirements

• Azure CLI
• Azure Container Registry (ACR)
• Azure AD (AAD)

Acceptance Criteria

• A bash script is available to provision ACR and service principals.
• A resource group is provisioned.
• An ACR resource is provisioned in the resource group.
• An AAD service principal is created with permissions to pull images from the ACR.
• An AAD service principal is created with permissions to push and pull images to and from the ACR.

Given script input parameter company 'contoso'
and script input parameter location 'eastus'
when the script is executed
then a resource group exists in East US with the name 'acr-eus-contoso',
and an ACR exists with the name 'acr-eus-contoso' and sku 'Standard' inside the resource group,
and an AAD service principal exists with the name 'acr-contoso-pull', scoped to the ACR resource, and role set to 'acrpull',
and an AAD service principal exists with the name 'acr-contoso-push', scoped to the ACR resource, and role set to 'acrpush'.

Notes

Team identified that this is also needed by bedrock project. There should be a second effort to reconcile how bedrock might consume what is implemented with this story. microsoft/bedrock#210

Story

As a dev, I would like the stories from sprint 1 to have automation coverage, so that i do not have to test them manually.

Acceptance Criteria

Tests are created that assert issues #4 are deployed according to acceptance criteria defined in each story.
The tests should prove they fail if the deployment does not match criteria
The tests should prove they pass if the deployment does match criteria

Update resource infrastructure to adhere to resource naming convention.

Note: This has a dependency on the test framework. PR.

Move the ARM templates in this repo to here.

Anticipating two sets of infrastructure templates, one for ARM and another for Terraform, organize the templates and scripts into folders .\arm and .\tf. This item is solely about the ARM templates though.

Description

As a developer, I can configure and deploy Azure App Service

For Cobalt, App Service should provide:

• App/container hosting
• Public IP mapped to container port (Dependency on Julio Research)
• Load balancing (Another User story for scale out config)

Acceptance Criteria

Reference: [Done-Done Checklist] (https://github.com/Microsoft/code-with-engineering-playbook/blob/master/Engineering/BestPractices/DoneDone.md)

• Service plan is already deployed before starting deployment of app service

• Key value pair for app service name and linux fx version

• open question: service aplan module depends on app service

• [ ]Output in readme with an example output

• VNET is an option for App Service. (No data block needed)

• No hardcoded variables in module config; everything set through Terraform template

• Usage referenced in Azure Simple template

• Screenshot showing resource provisioned in Azure Portal

Also, here are a few points that need to be addressed:

1. No hardcoded variables in module config; everything set through Terraform template

Resources

Tasks

Stories are intended to be completed in a single sprint; if task breakdown creates addition work then team should discuss promoting the Story to an Epic.
Reference: [Minimal Valuable Slices] (https://github.com/Microsoft/code-with-engineering-playbook/blob/master/Engineering/BestPractices/MinimalSlices.md)

Reference: [How to Write Better Tasks] (http://agilebutpragmatic.blogspot.com/2012/04/splitting-story-into-tasks-how-to-write.html)

Assignee should break down work into tasks here