Code Monkey home page Code Monkey logo

azure-redcap-paas's Introduction

REDCap Deployment on Azure

Overview

This repository provides you with the necessary resources and guidance to deploy the REDCap application on Microsoft’s Azure cloud platform. This allows you to leverage the power of cloud computing for your research data management needs.

This template automates the deployment of the REDCap solution into Azure using managed PaaS resources. The template assumes you are deploying a version of REDCap that supports direct connection to Azure Blob Storage. If you deploy an older version, deployment will succeed but you will need to manually provision NFS storage in Azure, and delete the new storage account. For NFS, consider:

Deployment Options

  • Manual deployment

    • For manual deployment process, please navigate here
  • CI/CD Deployment with GitHub

    • Information pending
  • CI/CD Deployment with Azure DevOps

    • Information pending

Details

This template automates the deployment of the REDCap solution into Azure using managed PaaS resources. The template assumes you are deploying a version of REDCap that supports direct connection to Azure Blob Storage. If you deploy an older version, deployment will succeed but you will need to manually provision NFS storage in Azure, and delete the new storage account. For NFS, consider:

To deploy the REDCap source to Azure App Service, you must supply your REDCap Community site credentials. The deployment automation will use them to pull the REDCap source directly from the community site.

NOTE: These values will be stored within the Azure App Service as configuration settings. Once your deployment has succeeded, you should navigate to your Azure App Service resource and delete or clear the values so that they aren't stored here.

Azure App Service

https://projectredcap.org/wp-content/resources/REDCapTechnicalOverview.pdf

  • The template deploys the following:
    • Azure Web App
    • Azure DB for MySQL (1)
    • Azure Storage Account
    • Key Vault
    • Private DNS zones
    • Virtual Network
    • Application Insights

(1) Review https://learn.microsoft.com/azure/mysql/flexible-server/concepts-service-tiers-storage for details on available features, regions, and pricing models for Azure DB for MySQL.

Setup

This template will automatically deploy the resources necessary to run REDCap in Azure using PaaS (Platform-as-a-Service) features.

IMPORTANT: The "Workload Name" you choose will be re-used as part of the storage, website, and MySQL database name. Make sure you don't use characters that will be rejected.

After the template is deployed, deployment automation will download the REDCap ZIP file you specify, and install it in your web app. It will then automatically update the database connection information in the app.

NOTE: The database will not be initialized; therefore, REDCap will not be usable until then. See the Post-Setup section below on how to initialize the database.

With the download and unzipping of REDCap application, the entire operation will take between 12-16 minutes.

If you need to connect to the MySQL database using the MySQL client, you will need to deploy a Virtual Machine with Bastion or AVD to the virtual network to run the client.

The database user name defaults to sqladmin and the password is a random string of 25 characters. The password is stored in Key Vault.

Post-Setup

After the deployment and installation of REDCap has completed, you will need to configure some database settings manually. The application gets deployed via Kudu which calls the deploy.sh script. After deployment, the postbuild.sh script will call REDCap's built-in capability to deploy the database schema. However, the configuration of the attachment storage to Azure Storage requires executing SQL statements that cannot be automated at this time. There is an install.sh file that contains the statements to be executed.

Once the source control deployment of REDCap has completed, you will need to SSH into the running container:

ssh

Execute the following command from the /home directory:

bash ./site/repository/scripts/bash/install.sh

ssh

Once you regain access to the console, you can navigate to the root of your app service and confirm everything shows green on the REDCap Configuration Check page - with the exception of CronJob status which you may have to manually invoke. If anything displays on that page in red or yellow, it is recommended that you perform a "Restart" of the Azure "App Service". This needs to be done due to the fact that some necessary server environment settings get changed after the initial deployment, but restarting the App Service will load the service with the intended settings.

Note about REDCap "Easy Upgade"

The "Easy Upgrade" feature in REDCap 8.11.0 and later is currently not supported when deploying a REDCap instance on Azure. Support for "Easy Upgrade" on Azure is expected to come at a later time in a future REDCap release.

Resources

Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://opensource.microsoft.com/cla/.

When you submit a pull request, a CLA-bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., label, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.

azure-redcap-paas's People

Contributors

bretthackermsft avatar kalalvishal avatar matheuscarboni avatar microsoft-github-policy-service[bot] avatar pauldotyu avatar svenaelterman avatar taylorr4 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

azure-redcap-paas's Issues

Unable to deploy: Microsoft.Web/sites/sourcecontrols conflict

Hello! I have been trying deploy REDCap (v8.11.8) using the Deploy using SendGrid template and my deployment is failing with this error:

{
    "status": "Failed",
    "error": {
        "code": "ResourceDeploymentFailure",
        "message": "The resource operation completed with terminal provisioning state 'Failed'."
    }
}

Screen Shot 2020-08-04 at 1 15 22 PM

I am new to Azure and would really appreciate some help. Please let me know if you need any additional information regarding my deployment.

Your deployment failed

Hi,

I have an error deploying RedCAP on Azure "Your deployment failed"

Error message is:

STATUS Conflict
STATUS MESSAGE {
"status": "Failed",
"error": {
"code": "ResourceDeploymentFailure",
"message": "L'opération de ressource s'est achevée avec l'état d'approvisionnement terminal « Failed »."
}
}
PROVISIONING STATE Failed
TIMESTAMP 4/26/2019, 5:11:03 PM
DURATION 39 seconds
TYPE Microsoft.Web/sites/sourcecontrols

Someone have an idea how to fix it?

May I download and join here the deployment details file?

Best regards,
Emmanuel

Create unique deployment resource names

The name of the main deployment is (mostly) unique based on a timestamp. Each child deployment uses the same name every time. When executing multiple deployments simultaneously, this leads to problems.

Admittedly, this is a rare condition that is expected to happen only in sandbox environments, but the fix is easy enough to implement.

Transferred from kalalvishal#38

Terraform: Update deprecated azurerm_app_service_plan resource to azurerm_service_plan

azurerm_app_service_plan is deprecated in version 3.x of the azurerm provider. The recommended guidance is to use the new azurerm_service_plan resource. Fortunately the update is relatively straightforward.

# the existing:
resource "azurerm_app_service_plan" "redcap" {
  name                = "${local.app_service_name}Plan"
  resource_group_name = azurerm_resource_group.redcap.name
  location            = azurerm_resource_group.redcap.location
  tags                = var.tags
  kind                = "Linux"
  reserved            = true

  sku {
    tier     = var.app_service_plan_tier
    size     = var.app_service_plan_size
    capacity = var.skuCapacity
  }
}

# becomes: 
resource "azurerm_service_plan" "redcap" {
  name                = "${local.app_service_name}Plan"
  resource_group_name = azurerm_resource_group.redcap.name
  location            = azurerm_resource_group.redcap.location
  os_type             = "Linux"
  sku_name            = var.app_service_plan_size
}

The app_service_plan_tier and skuCapacity arguments can then be removed.

Any existing projects updating from the one resource to the other will need to use terraform state move or terraform import to bring the existing resource under management of the new resource.

Terraform: Remove deprecated azurerm_app_service resource in favor of azurerm_linux_web_app & azurerm_windows_web_app

azurerm_app_service is deprecated in version 3.x of the azurerm provider. The recommended guidance is to use the new azurerm_linux_web_app & azurerm_windows_web_app resources. These can be implemented via parallel resources (one for linux, one for windows) enabled/disabled via count with a conditional based on the parent app service resource's type, like so:

resource "azurerm_linux_web_app" "redcap" {
  count                     = var.os_type == "Linux" ? 1 : 0
  ...
}

...
resource "azurerm_windows_web_app" "redcap" {
  count                     = var.os_type != "Linux" ? 1 : 0
  ...
}

Note this does (unfortunately) require all dependent resources to include the same conditional and also use a index, like so:

resource "azurerm_private_endpoint" "web_app" {
  ...

  private_service_connection {
    name                           = "example"
    private_connection_resource_id = var.os_type == "Linux" ? (
      azurerm_linux_web_app.redcap[0].id 
    ) : ( 
      azurerm_windows_web_app.redcap[0].id
    )
    subresource_names              = ["sites"]
    is_manual_connection           = false
  }
  ...

}

E-signature not prompting for AAD credentials

Hello,

E-signatures (an extension of the record locking/unlocking functionality) when toggled will prompt the user for their credentials, however, the credentials prompted for are not AAD credentials (we have Azure OAuth2 authentication method enabled).

2021-12-07_9-55-08

MySQL Version

We've used MySQL v8 on site and are trying to migrate to Azure. Only MySQL 5.6 & 5.7 are available. Can MySQL version 8 be added?

deployment broken (smtp relay option)

After a completed deployment in Azure with no completion errors i am getting a forbidden error, i have looked to initialize the database with the steps provided in the readme, but the fires were not under /home, they were under /home/site/repository and none of them could execute becuase some files were amiss (failed to open /home/install.sql)

I can find most of the installation artefacts under the site/repository, and i am using the latest build of RedCap (13.,1.35 LTS)

Problems with email

Hi!
We have deployed RedCAP on Azure resources with the ARM Template. Everything seems to work except the emails. REDCap, in the configuration check page, says that it is able to send emails but no email are sent.
The SMTP configurations are correct (tested with a mail client). Any idea on how we could investigate on the cause of the problem?

Support disabling sql_generate_invisible_primary_key via MySQL configuration entry (resolves MySQL 8.x issue as reported by REDCap database checks)

As reported in the REDCap Community forums, building REDCap from scratch when using the latest 8.x version of MySQL is (sporadically) resulting in the following error:
YOUR REDCAP DATABASE STRUCTURE IS INCORRECT

The solution is to set the sql_generate_invisible_primary_key value on the MySQL flexible server to 'OFF' prior to running the REDCap install script for the database.

Via Terraform these could be implemented via an azurerm_mysql_flexible_server_configuration resource:

resource "azurerm_mysql_flexible_server_configuration" "disable_invisible_primary_key" {
  resource_group_name = azurerm_resource_group.redcap.name
  server_name         = azurerm_mysql_flexible_server.redcap.name
  name                = "sql_generate_invisible_primary_key"
  value               = "OFF"
}

I have not yet tested whether this has an impact when using prior version of MySQL: if so it could include a count = startswith(var.mysqlVersion, "8") ? 1 : 0. Or even better such a conditional used in a locals block should add a map an argument to a merge() function such as suggested in #50 (preferred).

Support enabling audit logs on MySQL Flexible Server

To enable audit logs to a LAW in Azure, the following MySQL configuration entries are needed:
"audit_log_enabled" = "ON",
"audit_log_events" = "ADMIN,CONNECTION,DCL,DDL"

The latter may vary depend on what logging is desired from the server.

Via Terraform these could be implemented via a pair of azurerm_mysql_flexible_server_configuration resources:

resource "azurerm_mysql_flexible_server_configuration" "audit_log_enabled" {
  resource_group_name = azurerm_resource_group.redcap.name
  server_name         = azurerm_mysql_flexible_server.redcap.name
  name                = "audit_log_enabled"
  value               = "ON"
}

resource "azurerm_mysql_flexible_server_configuration" "audit_log_events" {
  resource_group_name = azurerm_resource_group.redcap.name
  server_name         = azurerm_mysql_flexible_server.redcap.name
  name                = "audit_log_events"
  value               = "ADMIN,CONNECTION,DCL,DDL"
}

These could either be enabled via an option variable in the variables.tf file (e.g. enable_audit_log), or even better such a variable could add the above defined in locals as an argument to a merge() function such as suggested in #50 (preferred).

This will also require a diagnostic setting resource and a LAW to house logs. In Terraform:

resource "azurerm_monitor_diagnostic_setting" "logging" {
  count                      = var.enable_audit_log
  name                       = "mds-mysql-redcap"
  target_resource_id         = azurerm_mysql_flexible_server.redcap.id
  log_analytics_workspace_id = var.log_analytics_workspace_id

  # https://learn.microsoft.com/en-us/azure/azure-monitor/essentials/resource-logs-categories#microsoftdbformysqlflexibleservers
  enabled_log {
    category = "MySQLAuditLogs"
  }

  metric {
    category = "AllMetrics"
  }
}

REDCap requirements for PEAR

Since this is for App Service (old name is Azure Website), do we still need to configure for PEAR package ? if yes, how we can enable it in Kudu ?

Use environment variable references in database.php

Currently, the deployment script dumps the database connection values in database.php. This means that after deployment, it's not possible to update those values. Moreover, those values are available in plain text in database.php.

We need to use something like $_ENV['db_username'].

Fix REDCap installation performance

Unzip takes too long on blob storage.

Considering mounting Premium file share and unzipping there, then copying files to /home/site

Alternatively, build a REDCap container image

Transferred from kalalvishal#5

Terraform: add support for arbitrary number of azurerm_mysql_flexible_server_configuration resources

Terraform supports MySQL Flexible server configuration using azurerm_mysql_flexible_server_configuration resources, one for each configuration entry. An addition to the variables.tf file with a supporting variable in addition to the configuration resource using a for_each loop would be ideal.

Example implementation (including locals for universal defaults):
variables.tf

variable "mysql_configuration_items" {
  description = "(Optional) Map of MySQL configurations to enable on the flexible server.  Defaults to `{}`"
  type        = map(string)
  default     = {}
}

main.tf
(see #51 & #52 for why these locals could be included as defaults)

locals {
  mysql_logging_configuration = var.enable_audit_log ? {
    "audit_log_enabled" = "ON",
    "audit_log_events" = "ADMIN,CONNECTION,DCL,DDL",
  } : {}
  mysql_default_configuration_items = merge( local.mysql_logging_configuration, {
    "sql_generate_invisible_primary_key" = "OFF",  # resolves recent issues reported by REDCap database checks when using MySQL v8
  }
}

...

resource "azurerm_mysql_flexible_server_configuration" "config_item" {
  for_each            = merge(local.mysql_default_configuration_items, var.mysql_configuration_items)
  resource_group_name = azurerm_resource_group.redcap.name
  name                = each.key
  server_name         = azurerm_mysql_flexible_server.redcap.name
  value               = each.value
}

While not as familiar with the other deployment options, I assume an analog could be created for those deployment options as well.

This will facilitate a number of other features, including enabling audit logging on the MySQL server, addressing the sql_generate_invisible_primary_key issue with MySQL 8.x (see REDCap community entries on that issue), and others.

Example usage in a tfvars file:

mysql_configuration_items = {
  "max_connections" = 255
}

Modify workload name array

Naming convention depends on nameModule and string array workloads that require the array order to match. Need to modify this into object or else in order to prevent changes to workload name from breaking

Transferred from kalalvishal#45

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.