Currently, many steps and file deletions are needed in order to use a non-custom image. Ideally, it should be done dynamically or based on a small set of centralized configuration.

We need to precise that the terraform workspace name does not contain - or _ characters. Otherwise, HANA installation will failed with a "invalid hostname" message.

Description:

Checklist of things need to be done before next release in GitHub.

Checklist

• fix output.tf - useful for openQA tests (#217 and #220)
• add monitoring for GCP (#219)
• add monitoring for AWS (#203)
• have the choice to disable monitoring on Azure (#182)
• ntpd issue on SLE12+ (#208)
• disable ha_exporter by default (#181)

Validation

Test runs should passed on all providers with pillar_examples/automatic/*.sls files and with default values

Validation with background = true (recommended value):

• Azure
• AWS
• GCP
• Libvirt

Validation with background = false (default value, and used for automated QA tests with openQA):

• Azure
• AWS
• GCP
• Libvirt

currenlty we are using pkg from suma, but we should use from ha-sap-factory repo.

So just copypack those pkg and update this repo to use them.

Descriptions:

todos doesn't belong to readme.md files.

I'm moving the todos form AWS to a github ISSUE because the todos will be removed from AWS

To Do for aws

• Investigate if it is possible to upload the images directly with terraform
• Check AWS documentation for Hana setup and add required resources. Current configuration works for build validation of new images, but lacks certain resources that are probably needed (Load Balancer, for example) for a complete setup of Hana in AWS.
• Find if it's possible to create more than one device with iscsi-formula.
• Add SLES12 compatibility for iscsi-formula.

https://github.com/SUSE/ha-sap-terraform-deployments/blob/master/azure/network.tf#L141

is still this valid ? Should we not support just the latest hana?

description:

I have left the monitoring image hardcoded to a specific image.

I think we could make this parametrizable or at least we should research.

I personally don't think we should change that often, so even a commit for updating them could be a valid solution.

  // TODO add variable later
  storage_image_reference {
    id        = azurerm_image.monitoring.0.id
    publisher = "SUSE"
    offer     = "SLES-SAP-BYOS"
    sku       = "15"
    version   = "2019.07.17"
  }

variable "running" {
  description = "whether this host should be turned on or off"
  default     = true
}

this variable is cosmetical to me. When we deploy a VM we want it always running.

We should keep our codebase KISS and remove it.

@nick-wang @ldevulder @alvarocarvajald any thouhgts on this? do you want to keep it? I would be in favor to remove it, since nobody use it.

descriptions:

This card will serve me as epic for organizing our epic and track some issues, order them.

Tasks

• investigate how to gather data

Problems:

dashboard related:

• if node1 reboots we loose all the HA metrics. (hawk-apiserver) since the dashboard is pointing to node1.

resource migrations of exporters

• if node1 reboots we should migrate the hanadb exporter to the node2 where the hanadb will live (since the node1 will have no data)

current situation:

everyone has already encountered this The problem is that this error doensn't anything useful for debugging and can be for real different reasons

Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/salt/utils/templates.py", line 392, in render_jinja_tmpl
    output = template.render(**decoded_context)
  File "/usr/lib/python3.6/site-packages/jinja2/asyncsupport.py", line 76, in render
    return original_render(self, *args, **kwargs)
  File "/usr/lib/python3.6/site-packages/jinja2/environment.py", line 1008, in render
    return self.environment.handle_exception(exc_info, True)
  File "/usr/lib/python3.6/site-packages/jinja2/environment.py", line 780, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/lib/python3.6/site-packages/jinja2/_compat.py", line 37, in reraise
    raise value.with_traceback(tb)
  File "<template>", line 23, in top-level template code
jinja2.exceptions.UndefinedError: 'dict object' has no attribute 'init'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/salt/utils/templates.py", line 169, in render_tmpl
    output = render_str(tmplstr, context, tmplpath)
  File "/usr/lib/python3.6/site-packages/salt/utils/templates.py", line 402, in render_jinja_tmpl
    buf=tmplstr)
salt.exceptions.SaltRenderError: Jinja variable 'dict object' has no attribute 'init'
[CRITICAL] Rendering SLS 'base:cluster' failed: Jinja variable 'dict object' has no attribute 'init'
local:
    Data failed to compile:
----------
    Rendering SLS 'base:cluster' failed: Jinja variable 'dict object' has no attribute 'init'

expected results:

we should research and find a better error, and research if we can error-out before or in a more precise way, controlling why this happen with a more isolated small error. this will help debug and UX

Right now, the monitoring stack is always deployed in Azure. This must be changed to work as in libvirt, configurable.
We should reuse the monitoring_enabled parameter as in libvirt,

this is followup from my last pr: #143

I found several issue with the cluster on azure.

hana_public_sku     = "15"
hana_public_version = "2019.03.30"

Given this images the nodes didn't deployed

currenlty only libvirt use module

We should unify all provider to same structure

Dependencies:

• we should do this only after terraform v12

Description:

The devel_mode grain and variable should be in future removed.

IT add to much behavioural differences.. E.g it can be really tricky to test.

In theory we should have just 1 way of doing things, avoiding each variable if it can be spared.

Terraform provides the capability of creating modules to better code organization and to create reusable components. See: Creating Modules.

Moving some of the components to modules could help to create more complexes deployments, like Scale-out, HANA with multi system replication, reusing the already existing code.

info:

We should remove un-used variables for keep codebase minimal.

This 3 variables at least 2 (add repo and add pkgs) aren't really needed for our use-cases.

The add-modules perhaps

description:

we have states called foo-bar and foo_bar2. We need to stick to the suggested upstream format and unify them all to have a convention.

Consider just having SAPCAR.EXE (Linux edition) and HANA SAR file in the upload.
Then get the automation to download and extract locally on VM from cloud storage.

Pros.
Keeps the Fileshare / S3 bucket nice and clean with just two files in (or perhaps can use BLOB storage).
Easy to upload (initially)
Easy to update with new/multiple versions. (ie. V1 or V2 etc)

Cons.
Inefficient - automation will have to extract each time - so will add to build time.

description:

we need to improve the way we handle the monitoring of port/adress exporters.

Right now we add a default value to the hosts.

But we can improve this mechanism further, adding a grain for example or other things to have more control of what we monitor (adress), and in which port

Followup discussion with @arbulu89

I think that we need to have a specific grains entry for the monitoring module, something like ['address1:port1', 'address2:port2...]

description:

Question regarding HA monitoring with Grafana. Are we already considering the dashboards to work with several clusters? I'm saying that because when NW is deployed together with HANA, we will have 2 HA clusters. Just wondering....

This issiue is a followup to this question whic might be legitim to keep in mind.

In future we can close this issue or move it elsewhere

Problem:

I don't have much bandwith for deploying to the central deplyement server.

I deploy from France terraform apply to NUE KVM server where we can deploy.

Deploying this way take times, especially for copying from France to NUE the images.

( there is no way that we can do it on the terraform-libvirt-kvm, I have already researched).

Currently if the base-image exists, we prompt out and error out that resource already exists.

We need to research a way to create a global shared image that it is created, and used in a cached way

Technical implementation:

This problem can be solved in many ways.

1. create a terraform null_resource which execute a script for checking that the image already exist and is the latest one, otherwise recreate/download the lates.

2. it could be also a crontab job running on the server which download and create the shared img and the terraform files depends explitely from that.

For solution 2, we should create like a special golum directory terraform files, which we state that we depends from that img beeing create by x custom script

3. an other implementations ..

We should aim to handle repositories more generic and take inspiration of sumaform way, where repos are files and they are added automatically by detecting the OS.

Originally posted by @MalloZup in #103

Applying the GCP salted configuration with init_type = "skip-hana", the deployment fails with the following error:

local:
    Data failed to compile:
----------
    Rendering SLS 'base:hana_node' failed: Jinja variable 'dict object' has no attribute 'monitoring_enabled'

Checked also with init_type = "all", and the same issue is observed.

I set the variable monitoring_enabled to false in my terraform.tfvars and still terraform continue to create the monitoring module, Also asking to specify "monitoring_srv_ip"

NTP was replaced by Chrony on SLE-15 and higher code streams, so the deployments should be adjusted to configure Chrony instead of NTP.

NTP is still available on the sle-module-legacy and it is supported, but as the legacy module is not supported for the entire product lifecycle, this switch should be done ASAP.

Now that we have lots of Salt files around it would be nice to have a Travis check for these.

Description:

we have rules of azure load balance that need to be renamed correctly.

https://github.com/SUSE/ha-sap-terraform-deployments/blob/master/azure/network.tf#L71

we should not give number names but rather service names which are understanble by human.

lb_30313 doesn't say anything

Task

rename all the rule with a service name

Problem/desc:

we have currenlty name-prefix as tfvars.

https://github.com/SUSE/ha-sap-terraform-deployments/search?q=name_prefix&unscoped_q=name_prefix

We don't need this variable since we can take it form the workspace name as we did in past.

e.g see: https://github.com/SUSE/ha-sap-terraform-deployments/blob/master/libvirt/terraform/modules/base/main.tf#L6

Research if we can push our dashboard (when complete more or less to grafana) or other mediums.

I have noticed that the README in the salt folder doesn't make any sense.

We should update it to show the relevant information about the folder.
Otherwise, it would make actually more sense to delete the doc file

description:

This issue is a follow-up.

We should perhaps research how to improve hcl syntax. I will try to search if I can do it, but I have quite already digged and the current syntax is the ugly cleanest one :grin

so it can be tricky to find a better one

details

What would happen here if we have 2 disk in the additional_disk list?
The solution looks quite strange (maybe there is not any better).
Anyway, we could improve it in the future as well, I don't really mind if it works

Originally posted by @arbulu89 in #116

The Azure team also have a terraform based project to deploy SAP Solutions. Azure/sap-hana.

They are also already using modules and both projects have the same purpose. So basing the Azure deployment on that project could benefit both projects as the underlying terraform deployment would be maintained and evolved by more people.

I'm doing some test to be able to execute our Public Cloud test within openQA (to have tests automatically triggered for each new build) and a minor change may be needed to be able to reuse some code from qa-kernel team: they are using 'EC2' for AWS and 'GCE' for 'GCP' (these acronyms are valid), and in our code we use 'aws' and 'gcp' for the directory names.

So could be either rename the directories or add links?

It might be worth commenting in the 'How to use section' that the image.tf file needs to be removed if using public images.

(There is a section on renaming the virtualmachines.tf-publicimg, but not the fact that image.tf is not needed either).

If future versions will be designed for 'public images first' then you can ignore this.

We have several grains which are not coherent along providers and codebase.

We can remove some of them or merge them in order to have less variable as possible

research:

propose and research which grain can be removed or merged togheter

Tasks: (ordered by prio)

• #152 (add_pkg/add_repo)
• #93
• #151 (devel_mode)

Also unreleated but is part of this refactoring:

#105 (use unique syntax - vs _ etc)

Problem Description

In our terraform.tfvars we ask for REG_CODE and REG_email

This is kind tedious since it might force people/newbie to create the reg-credentials etc.

Having a token mechanism is more straightforward mechanism and we could in theory re-use the same token everywhere also for CI.

Technical consideraitons:

• Investigate if the suseconnect CLI can pass a token for Registration etc.
  

  ha-sap-terraform-deployments/salt/default/registration.sls

  Line 5 in 8876221

  - name: /usr/bin/SUSEConnect -r {{ grains['reg_code'] }} {{ ("-e " ~ grains['reg_email']) if grains['reg_email'] != "" else "" }}

(SUSE/connect#416)

• Consider if we want to keep the 2 methods or having only 1 replacing the others.

At least in libvirt provider README file, the ssh keys creation step is not mentioned in the quickstart chapter.

The use would need to run:

cd salt/hana_node/files
mkdir sshkeys
cd sshkeys
ssh-keygen # Create ssh keys with the name id_rsa_cluster and without passphrase

we should find out a better way of doing this

Related to #133. Right now, there is a special variable monitored_hosts telling which hosts will be monitored. A better approach here is to enable monitoring only for the primary node, assign a virtual ip for each exporter and manage the exporters as a resource managed by HA stack. This will allow us to remove monitored_hosts variable.

Hello,

does SUSE have a certified solution for Standalone Enqueue Server 2 that's a part of NW 7.51? (https://help.sap.com/viewer/cff8531bc1d9416d91bb6781e628d4e0/1709%20001/en-US/a2d5c422cfff4fc286bc571965451093.html)

I didn't find any documentation or certification details.

thanks,
Ravi

Here's the Changelog with all the syntax and cli changes:
https://github.com/hashicorp/terraform/blob/v0.12.0/CHANGELOG.md

The upgrade guide with a link to a tool to automate most of the process:
https://www.terraform.io/upgrade-guides/0-12.html

We also need to wait for the openSUSE packagers to ship updated versions because of our use of the Terraform libvirt provider:
http://download.opensuse.org/repositories/systemsmanagement:/terraform/

desc:

this is mostly a reminder for future-me.

the shared-resource variable for libvirt doesn't make sense.
Remove it to achieve more coherence in codebase

desc:

We are using variables instead of a map for input paramters.

Originally the problem was that terraform has problem with the map, I suspect it was a terraform internal issue.

We might need to investiagate on this and bring them back if needed

historic:

Honestly, I preferred the map kind of data for this variable. But I don't know how it behaves with terraform 12.
Anyway, It's just a comment

Originally posted by @arbulu89 in #143

Right now, the ha_exporter is installed by default, without considering if the monitoring stack is created or not.
In order to align all the component, the ha_exporter should be only installed together with the whole monitoring stack.
What do you think?
For that, we should just set this value to False in the cloud_providers (aws and gcp don't even have monitoring stack) and in the automatic pillar example depending on the monitoring_enabled parameter.

Examples:
https://github.com/SUSE/ha-sap-terraform-deployments/blob/master/pillar_examples/aws/cluster.sls#L13
https://github.com/SUSE/ha-sap-terraform-deployments/blob/master/pillar_examples/automatic/cluster.sls#L29

In fact, we should take advantage, and do the same with the hanadb_exporter (only enable according this monitoring_enabled parameter.

We should document in the 1st readme section the different global modes we allow.

this mode each provider will implement them

devel_mode
qa_mode

Problem/Description:

We need to release a new GitHub release before we migrate to terraformV12 and do refactoring with the new HCL lang.

In this way we could have 2 different releases and events for keeping an historic track.

tasks:

• release with github.

notes:

I don't think we need much work on the release itself. Anyone could pick this up.

I wrote this card a promemoria.

Currently, the salted configuration for GCP doesn't support public images, so it's useful only for testing.

The non-salted configuration supports the use_custom_image variable for this.

description:

This card is more a meta-tracker for issue I found in azure. To keep track of them.

I wil refine the card step by step and update it.

tasks

• add similar mechanism with tfvars like libvirt for having same 1:1 parity. this should be the only was to give inputs.
• add monitoring
• Move to modules
• improve doc
• find better name for this: https://github.com/SUSE/ha-sap-terraform-deployments/blob/master/azure/resources.tf#L6
• naming of the resources are bad myas ,myroutes etc this prefix should be removed

The gcp documentation is not that bad but we need more hints about the gcp_credential file creation for newbies

we need a way to improve the doc, especially we need to provide some hints and clear steps for a new user who want to create a new gpc credentials file

https://github.com/SUSE/ha-sap-terraform-deployments/tree/master/gcp/terraform_salt#prerequisites

# Credentials file for GCP gcp_credentials_file = "/home/brother-newbie/SUSE/cloud/foo.json"

Right now, ports are hard coded in the prometheus.yml file. The best option would be extracting those from the pillar files (clsuter.sls and hana.sls) when monitoring is enabled.