OCP 3.7 cluster deployment is failing while processing "deployOpenshift" about openshift-container-platform HOT 8 CLOSED

Give me a few days to look into this.

from openshift-container-platform.

We have fixed the issue by doing the following changes:

Step 1 : In file /usr/share/ansible/openshift-ansible/roles/openshift_sanitize_inventory/tasks/deprecations.yml, at line number 10
no_log: true is changed to no_log: false.

Step 2 : We have run the ansible-playbook again.
Now we are able to see the error:

failed! => {"msg": "template error while templating string: unexpected char u'f' at 3. string: {{xxxxxx-ca46-466f-xxxx-0fb3a3fxxxxx}}"}

Step 3: By removing the double braces "{{ }}" from the file /etc/ansible/hosts line numbers 30,31,32 and 33 this resolves the error.

from openshift-container-platform.

Hi,
While deploying Openshift Container Platform-3.10,

We are facing an error as" “VM has reported a failure when processing extension deployOpenShift”.

Can somebody look into this issue. it would be helpful. Thanks in advance..!!

Please find the below detailed information about parameters and error.

Parameters we used:
Master VM size: Standard_E2s_v3
Node VM size : Standard_DS4_v2
Infra VM size : Standard_DS4_v2
CNS VM size : Standard_E4s_v3
masterInstanceCount : 3
nodeInstanceCount: 3
InfraInstanceCount: 3
CNSInstanceCount : 3 ( CNS is Enabled )
dataDiskSize : 128 GB
enableAzure: True
Enable Metrics: True
Enable Logging : True

Error we received while deploying through ARM Template:
{"code":"DeploymentFailed","message":"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-debug for usage details.","details":[{"code":"Conflict","message":"{\r\n "status": "Failed",\r\n "error": {\r\n "code": "ResourceDeploymentFailure",\r\n "message": "The resource operation completed with terminal provisioning state 'Failed'.",\r\n "details": [\r\n {\r\n "code": "DeploymentFailed",\r\n "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-debug for usage details.",\r\n "details": [\r\n {\r\n "code": "Conflict",\r\n "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource operation completed with terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"VMExtensionProvisioningError\",\r\n \"message\": \"VM has reported a failure when processing extension 'deployOpenShift'. Error message: \\\"Enable failed: failed to execute command: command terminated with exit status=2\\n[stdout]\\nvice-broker/key.pem -CAcreateserial -out /etc/origin/ansible-service-broker/client.pem -days 1024'})\\n\\nTASK [ansible_service_broker : set_fact] ***************************************\\nok: [ocpcluster-master-0]\\n\\nTASK [ansible_service_broker : Read in certs for etcd] *************************\\nok: [ocpcluster-master-0] => (item=cert.pem)\\nok: [ocpcluster-master-0] => (item=client.pem)\\nok: [ocpcluster-master-0] => (item=client.key)\\n\\nTASK [ansible_service_broker : set_fact] ***************************************\\nok: [ocpcluster-master-0]\\n\\nTASK [ansible_service_broker : create openshift-ansible-service-broker project] ***\\nchanged: [ocpcluster-master-0]\\n\\nTASK [ansible_service_broker : create ansible-service-broker serviceaccount] ***\\nchanged: [ocpcluster-master-0]\\n\\nTASK [ansible_service_broker : create ansible-service-broker client serviceaccount] ***\\nchanged: [ocpcluster-master-0]\\n\\nTASK [ansible_service_broker : Create asb-auth cluster role] *******************\\nchanged: [ocpcluster-master-0]\\n\\nTASK [ansible_service_broker : Create asb-access cluster role] *****************\\nchanged: [ocpcluster-master-0]\\n\\nTASK [ansible_service_broker : Bind admin cluster-role to asb serviceaccount] ***\\nchanged: [ocpcluster-master-0]\\n\\nTASK [ansible_service_broker : Bind auth cluster role to asb service account] ***\\nchanged: [ocpcluster-master-0]\\n\\nTASK [ansible_service_broker : Bind asb-access role to asb-client service account] ***\\nchanged: [ocpcluster-master-0]\\n\\nTASK [ansible_service_broker : create asb-client token secret] *****************\\nchanged: [ocpcluster-master-0]\\n\\nTASK [ansible_service_broker : oc_secret] **************************************\\nok: [ocpcluster-master-0]\\n\\nTASK [ansible_service_broker : set_fact] ***************************************\\nok: [ocpcluster-master-0]\\n\\nTASK [ansible_service_broker : Create custom resource definitions for asb] *****\\nchanged: [ocpcluster-master-0] => (item=/usr/share/ansible/openshift-ansible/roles/ansible_service_broker/files/bundlebindings.automationbroker.io.yaml)\\nchanged: [ocpcluster-master-0] => (item=/usr/share/ansible/openshift-ansible/roles/ansible_service_broker/files/bundleinstances.automationbroker.io.yaml)\\nfailed: [ocpcluster-master-0] (item=/usr/share/ansible/openshift-ansible/roles/ansible_service_broker/files/bundles.automationbroker.io.yaml) => {\\\"changed\\\": false, \\\"item\\\": \\\"/usr/share/ansible/openshift-ansible/roles/ansible_service_broker/files/bundles.automationbroker.io.yaml\\\", \\\"msg\\\": {\\\"cmd\\\": \\\"/bin/oc create -f /tmp/bundles.automationbroker.io-jKhC5w -n default\\\", \\\"results\\\": {}, \\\"returncode\\\": 1, \\\"stderr\\\": \\\"Error from server: error when creating \\\\\\\"/tmp/bundles.automationbroker.io-jKhC5w\\\\\\\": etcdserver: request timed out\\\\n\\\", \\\"stdout\\\": \\\"\\\"}}\\n\\nPLAY RECAP *********************************************************************\\nlocalhost : ok=12 changed=0 unreachable=0 failed=0 \\nocpcluster-cns-0 : ok=0 changed=0 unreachable=0 failed=0 \\nocpcluster-cns-1 : ok=0 changed=0 unreachable=0 failed=0 \\nocpcluster-cns-2 : ok=0 changed=0 unreachable=0 failed=0 \\nocpcluster-infra-0 : ok=0 changed=0 unreachable=0 failed=0 \\nocpcluster-infra-1 : ok=0 changed=0 unreachable=0 failed=0 \\nocpcluster-infra-2 : ok=0 changed=0 unreachable=0 failed=0 \\nocpcluster-master-0 : ok=113 changed=37 unreachable=0 failed=1 \\nocpcluster-master-1 : ok=35 changed=1 unreachable=0 failed=0 \\nocpcluster-master-2 : ok=35 changed=1 unreachable=0 failed=0 \\nocpcluster-node-0 : ok=0 changed=0 unreachable=0 failed=0 \\nocpcluster-node-1 : ok=0 changed=0 unreachable=0 failed=0 \\nocpcluster-node-2 : ok=0 changed=0 unreachable=0 failed=0 \\n\\n\\nINSTALLER STATUS ***************************************************************\\nInitialization : Complete (0:00:52)\\nService Catalog Install : In Progress (0:01:34)\\n\\tThis phase can be restarted by running: playbooks/openshift-service-catalog/config.yml\\n\\n[stderr]\\n % Total % Received % Xferd Average Speed Time Time Time Current\\n Dload Upload Total Spent Left Speed\\n\\r 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0\\r100 6 100 6 0 0 1004 0 --:--:-- --:--:-- --:--:-- 1200\\nWARNING: The installed extension 'storage-preview' is in preview.\\nWARNING: The behavior of this command has been altered by the following extension: storage-preview\\n# ocpcluster-cns-0:22 SSH-2.0-OpenSSH_7.4\\n# ocpcluster-cns-0:22 SSH-2.0-OpenSSH_7.4\\n# ocpcluster-cns-0:22 SSH-2.0-OpenSSH_7.4\\nWarning: Permanently added the ECDSA host key for IP address '10.1.0.4' to the list of known hosts.\\r\\n# ocpcluster-cns-1:22 SSH-2.0-OpenSSH_7.4\\n# ocpcluster-cns-1:22 SSH-2.0-OpenSSH_7.4\\n# ocpcluster-cns-1:22 SSH-2.0-OpenSSH_7.4\\nWarning: Permanently added the ECDSA host key for IP address '10.1.0.5' to the list of known hosts.\\r\\n# ocpcluster-cns-2:22 SSH-2.0-OpenSSH_7.4\\n# ocpcluster-cns-2:22 SSH-2.0-OpenSSH_7.4\\n# ocpcluster-cns-2:22 SSH-2.0-OpenSSH_7.4\\nWarning: Permanently added the ECDSA host key for IP address '10.1.0.6' to the list of known hosts.\\r\\n [WARNING]: Could not match supplied host pattern, ignoring: oo_lb_to_config\\n [WARNING]: Could not match supplied host pattern, ignoring: oo_nfs_to_config\\n [WARNING]: Consider using the yum, dnf or zypper module rather than running\\nrpm. If you need to use command because yum, dnf or zypper is insufficient you\\ncan add warn=False to this command task or set command_warnings=False in\\nansible.cfg to get rid of this message.\\n [WARNING]: Could not match supplied host pattern, ignoring:\\noo_hosts_containerized_managed_true\\n [WARNING]: Could not match supplied host pattern, ignoring: oo_lb_to_config\\n [WARNING]: Could not match supplied host pattern, ignoring: oo_nfs_to_config\\n [WARNING]: Could not match supplied host pattern, ignoring:\\noo_exclude_bootstrapped_nodes\\n [WARNING]: Consider using the file module with mode rather than running chmod.\\nIf you need to use command because file is insufficient you can add warn=False\\nto this command task or set command_warnings=False in ansible.cfg to get rid of\\nthis message.\\n [WARNING]: Consider using the unarchive module rather than running tar. If\\nyou need to use command because unarchive is insufficient you can add\\nwarn=False to this command task or set command_warnings=False in ansible.cfg to\\nget rid of this message.\\nWarning: Permanently added 'ocpcluster-master-0,10.1.0.11' (ECDSA) to the list of known hosts.\\r\\n [WARNING]: Could not match supplied host pattern, ignoring: oo_lb_to_config\\n [WARNING]: Could not match supplied host pattern, ignoring: oo_nfs_to_config\\n [WARNING]: Could not create retry file '/usr/share/ansible/openshift-\\nansible/playbooks/openshift-service-catalog/config.retry'. [Errno 13]\\nPermission denied: u'/usr/share/ansible/openshift-ansible/playbooks/openshift-\\nservice-catalog/config.retry'\\n\\\".\"\r\n }\r\n ]\r\n }\r\n}"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}"}]}

from openshift-container-platform.

Hi

We have overcome the issue with following changes while deploying.

Branch : release 3.10
VM size : Defaults
Metrics : false
Logging : false
CNS : true

can someone will help with this please,
As we disabled Metrics and Logging while deployment using the Template, is there any way to create Metrics and Logging after post-deployment..?

from openshift-container-platform.

Hi

We have tried to deploy metrics and logging after post-deployment and followed the steps which are mentioned in the below URL:
https://github.com/MicrosoftDocs/azure-docs/blob/master/articles/virtual-machines/linux/openshift-post-deployment.md

But we are getting the errors as :

TASK [openshift_sanitize_inventory : include_tasks] *********************************************************************************************************
included: /usr/share/ansible/openshift-ansible/roles/openshift_sanitize_inventory/tasks/deprecations.yml for ocpcluster-master-0, ocpcluster-master-1, ocpcluster-master-2

TASK [openshift_sanitize_inventory : Check for usage of deprecated variables] *******************************************************************************
fatal: [ocpcluster-master-0]: FAILED! => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result"}
fatal: [ocpcluster-master-1]: FAILED! => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result"}
fatal: [ocpcluster-master-2]: FAILED! => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result"}
to retry, use: --limit @/usr/share/ansible/openshift-ansible/playbooks/openshift-logging/config.retry

PLAY RECAP **************************************************************************************************************************************************
localhost : ok=12 changed=0 unreachable=0 failed=0
ocpcluster-master-0 : ok=4 changed=0 unreachable=0 failed=1
ocpcluster-master-1 : ok=3 changed=0 unreachable=0 failed=1
ocpcluster-master-2 : ok=3 changed=0 unreachable=0 failed=1

INSTALLER STATUS ********************************************************************************************************************************************
Initialization : In Progress (0:00:05)

please help us with this...Thanks in advance..!!

from openshift-container-platform.

@MeeraRepo Can you please tell me the specific line in the deployOpenShift.sh file? Thanks.

from openshift-container-platform.

@haroldwongms We haven't changed anything in the deployOpenShift.sh file. And used the ARM template as it is, but while deploying we disabled the Metrics and Logging.
Post-deployment we have tried to install Metrics and Logging. and we have done the above changes in the file /etc/ansible/hosts on Bastion VM.

from openshift-container-platform.

This should be fixed for 3.10 and 3.11. No longer maintaining releases older than 3.10.

from openshift-container-platform.