The following procedure is intended to create VM's from an OVA template booting with static IP's when the DHCP server can not reserve the IP addresses.
OCP requires that all DNS configurations be in place. VMWare requires that the DHCP assign the correct IPs to the VM. Since many real installations require the coordination with different teams in an organization, many times we don't have control of DNS, DHCP or Loadbalancer configurations.
Sometimes we need to do a "bare metal" installation over vmware to set the network configuration with kernel parameters at boot (ip, gateway, nameserver, etc.).
The coreos documentation explain how to create configurations using ignition files. I created a python script to put the network configuration using the ignition files created by the openshift-install program.
When the VM boots, it will take the first IP provided by the DHCP server (probably will not be the IP set by the script). (...dhclient[836]: bound to 10.56.240.99 -- renewal in 18253 seconds.)
At the end of the boot process the VM will take the IP provided by the ignition file.
- Terraform latest version (v0.12.24) you can download here
- Have the ignitions files already created (bootstrap.ign, master.ign, worker.ign).
- Copy the 3 ignitions files to your webserver.
- Download the OVA image from the mirror repository to the
terraform/
folder. - Optional
install-config.yaml
file. - Optional: The fist section of script
go.sh
create the ignition files and copy to the webserver, if you already did this process you can delete that part and start from the Terraforn deployment section of the script.
If you want to create all running the go.sh
script.
-
Place the
install-config.yaml
in the root of this git repo. -
Update the
go.sh
script with the path or webserver to copy the ignition files to the webserver.
scp deploy/*.ign [email protected]:/var/www/html/
ssh [email protected] 'chmod 644 /var/www/html/*.ign'
# cp -f deploy/*.ign /var/www/html/
# chmod 644 chmod 644 /var/www/html/*.ign
- Edit the file
terraform/vars/common.tfvars
with the values of your installation; vmware configuration and Openshift configuration.
## VSPHERE CONFIGURATIONS, SHOULD BE THE SAME IN YOU install-config.yaml
vsphere_user = "[email protected]"
vsphere_password = "Password123!"
vsphere_server = "vcenter.sni.com.mx"
#
datacenter = "BHM"
datastore = "SAS-6K"
network = "VM Network"
resource_pool = "Resources"
host = "esxi67.sni.com.mx"
.
.
.
- Extract the contents from the OVA file to the
terraform/
folder, wi will get 2 files:coreos.ovf
anddisk.vmdk
.
../terraform$ tar xvf rhcos-4.4.3-x86_64-vmware.x86_64.ova
coreos.ovf
disk.vmdk
- Run the script
go.sh
, and wait to finish.
terraform$ ./go.sh
+++ dirname ./go.sh
++ cd .
++ pwd
+ DIR=/home/alex/work/RedHat/ocp4-vmware-terraform
+ rm -fr deploy
+ mkdir deploy
+ cp install-config.yaml deploy/
+ openshift-install create manifests --dir=deploy
INFO Consuming Install Config from target directory
WARNING Making control-plane schedulable by setting MastersSchedulable to true for Scheduler cluster settings
+ sed -i s/true/false/g deploy/manifests/cluster-scheduler-02-config.yml
+ openshift-install create ignition-configs --dir=deploy
INFO Consuming OpenShift Install (Manifests) from target directory
INFO Consuming Openshift Manifests from target directory
INFO Consuming Master Machines from target directory
INFO Consuming Worker Machines from target directory
INFO Consuming Common Manifests from target directory
+ scp deploy/bootstrap.ign deploy/master.ign deploy/worker.ign [email protected]:/var/www/html/
Warning: Permanently added 'bastion.ocp4.sni.com.mx,10.56.241.10' (ECDSA) to the list of known hosts.
bootstrap.ign 100% 299KB 20.5MB/s 00:00
master.ign 100% 1820 994.2KB/s 00:00
worker.ign 100% 1820 1.1MB/s 00:00
+ ssh [email protected] 'chmod 644 /var/www/html/*.ign'
Warning: Permanently added 'bastion.ocp4.sni.com.mx,10.56.241.10' (ECDSA) to the list of known hosts.
- Continue with the oc commands to complete the installation, approving CRS, etc. etc.
You can easily destroy the bootstrap once you don't need it anymore.
alex@:/../terraform $ ./destroy --target=module.bootstrap
data.vsphere_datacenter.dc: Refreshing state...
data.vsphere_host.esxi67: Refreshing state...
Plan: 0 to add, 0 to change, 1 to destroy.
.
Warning: Resource targeting is in effect
.
Do you really want to destroy all resources?
Terraform will destroy all your managed infrastructure, as shown above.
There is no undo. Only 'yes' will be accepted to confirm.
.
Enter a value: yes
.
module.bootstrap.vsphere_virtual_machine.clone[0]: Destroying... [id=423147d7-b7ff-11d2-1762-a2e10dd37584]
module.bootstrap.vsphere_virtual_machine.clone[0]: Destruction complete after 9s
.
Destroy complete! Resources: 1 destroyed.
Or you can destroy everything
alex@:/../terraform $ ./destroy
Plan: 0 to add, 0 to change, 4 to destroy.
.
Do you really want to destroy all resources?
Terraform will destroy all your managed infrastructure, as shown above.
There is no undo. Only 'yes' will be accepted to confirm.
.
Enter a value: yes
module.master.vsphere_virtual_machine.clone[1]: Destroying... [id=4231f92a-08c9-6afe-7f0c-b17105e7991a]
module.master.vsphere_virtual_machine.clone[0]: Destroying... [id=4231da99-3018-40dd-2532-157990aea86f]
module.master.vsphere_virtual_machine.clone[2]: Destroying... [id=423188fc-571c-f243-367c-c6edaf8906b2]
module.master.vsphere_virtual_machine.clone[2]: Destruction complete after 4s
module.master.vsphere_virtual_machine.clone[1]: Destruction complete after 4s
module.master.vsphere_virtual_machine.clone[0]: Destruction complete after 5s
vsphere_folder.cluster: Destroying... [id=group-v185]
vsphere_folder.cluster: Destruction complete after 0s
.
Destroy complete! Resources: 4 destroyed.
References
- https://docs.openshift.com/container-platform/4.3/installing/installing_vsphere/installing-vsphere.html
- https://docs.openshift.com/container-platform/4.3/installing/installing_bare_metal/installing-bare-metal.html
- https://www.terraform.io/
- https://github.com/terraform-providers/terraform-provider-vsphere
- https://coreos.com/ignition/docs/latest/network-configuration.html
- https://coreos.com/ignition/docs/latest/examples.html