Automation on Elastic Cloud Server Provisioning using a created image from Cloud Backup and Recovery
At the beginning, customer planned to use the existing server in Huawei Cloud Singapore region to perform data restoration from Cloud Backup and Recovery (CBR) and use Server Migration Service (SMS) to migrate restored Elastic Cloud Server (ECS) to CAE Malaysia. Unfortunately, all the existing server has been destroyed by customer and the proposed plan is not going to work anymore. The only solution for CBR migration is to create a Full ECS private image from CBR backup and use created image to provision a new ECS, then only perform migration using SMS to CAE.
However, the process of creating a Full-ECS image from CBR backup and creating ECS from the created images need to be done manually by customers using the Huawei Cloud console. Customers have to repeat the step from creating an image from CBR backup to creating ECS with the image created multiple times because there are 134 backups that need to be restored. This is very time-consuming and might have human errors.
In this artifact, we are going to achieve the below objectives:
- To automate the process of creating a Full-ECS image from Cloud Backup and Recovery (CBR) and creating an Elastic Cloud Server (ECS) from the created image using Huawei Python SDK.
- To automate the process of resource cleaning using Huawei Python SDK.
- To develop an automated program to reduce time consumption.
Figure 2.0: image creation from CBR and ECS provisioning
Figure 2.1: image and ECS cleaning
Before getting started, complete the following prerequisites:
- To use Huawei Cloud Python SDK, you must have a Huawei Cloud account as well as the Access Key and Secret Key of the Huawei Cloud account. You can create an Access Key in the Huawei Cloud console. You can obtain the 🔑 AK/SK pair for your account at My credentials > Access keys > Create Access key.
Figure 3.0: Example of how to create Access Key
- To use Huawei Cloud Python SDK to access the APIs of a specific service, please make sure you have activated the service in the Huawei Cloud console if needed.
- Huawei Cloud Python SDK requires Python 3.3 or later, run the command python --version to check the version of Python. If Python 3.3 or later is installed. skip step 4.
Figure 3.1: Command Prompt check Python version
- Install Python 3 and add Python to PATH. Open your browser and navigate to the Downloads for Windows section of the official Python website. Navigate to the Using Python on Windows section for steps to install Python in the Windows environment.
- Please note that this program is only applicable for Windows environments only for now.
Before starting to automate the process of creating images and Elastic Cloud Server (ECS) in Huawei Cloud, some steps need to be done:
- Download Source File
- Install Python Packages
- Configure Environment Variable
- Click hwc_automation.zip to download the Source Code Zip archive file. Extract the Zip archive file after the download is complete.
- Check File Content.docx for the file content.
- Open the hwc_automation folder you have downloaded and extracted it in the previous step.
- Inside the folder, find install.bat and double-click to run it.
- This batch file will install all the Python Packages defined in requirement.txt.
- Python packages in requirement.txt
- huaweicloudsdkecs
- huaweicloudsdkims
- huaweicloudsdksmn
- huaweicloudsdksms
- requests
- python-dotenv
- openpyxl
- pandas
- Environment variables need to be declared first before proceeding to create Image, create ECS, delete Image, and Delete ECS.
- Inside the hwc_automation folder, find config.bat and double-click to run it.
- This batch file will start Command Prompt and ask the user to input parameters as prompted.
- The parameters required and steps to obtain as shown below:
Parameter | Steps |
---|---|
Access Key & Secret Key (AK/SK pair) |
1. On the console homepage, hover over the username in the upper right corner and choose My Credentials from the drop-down list. 2. Choose Access Key from the navigation tab at the left and click Create Access Key. 3. Click Download. 4. Open the credentials.csv you have downloaded to obtain AK/SK pair. |
Admin Pass |
1. On the console homepage, under Management & Governance, click Identity and Access Management. 2. In the navigation pane on the left, choose Account Security. 3. On the Account Security Settings page, select Password Policy in the navigation pane. 4. Follow the Password Policy to set Admin Pass. Admin Pass is required to create ECS and login into the ECS. |
VPC ID |
1. On the console homepage, under Network, click Virtual Private Cloud. 2. The VPC ID will appear once the cursor hovers over the VPC name. 3. After that, click the copy to clipboard icon beside the VPC ID to copy it. |
Subnet ID |
1. On the console homepage, under Network, click Virtual Private Cloud. 2. On the navigation pane at the left, click Subnets. 3. On the displayed Subnets page, locate your mouse cursor on the Subnets name. 4. The subnet ID will appear once the cursor hovers over the subnet name. After that, click the copy to clipboard icon beside the subnet ID to copy it. |
Figure 4.0: Command Prompt to input configuration parameters.
Figure 4.1: Example to obtain VPC ID
Figure 4.2: Example to obtain subnet ID
- Parameter input by the user will save in the config.env environment variable file. You may open the hwc_automation > Code > config.env file to verify that the parameters are correctly declared.
In this demo, we are going to automate the process of creating a Full-ECS image from Cloud Backup and Recovery (CBR) backup and creating an Elastic Cloud Server (ECS) from the created images. Step by step is as below:
- Inside the hwc_automation folder, find create_image_and_ecs.bat and double click to run it.
- This batch file will start a command prompt to show the number of sheets of the previous running and ask the user to input the number of sheets that need to be used in the current running as shown in Figure 5.0.
Note: None in the figure below indicate that the program never run before.
- The input is used to determine which sheet in ids_list.xlsx is to be used in creating images and servers. For example, if we input 1, sheet1 in ids_list.xlsx will be used while if we input 2, sheet2 will be used accordingly.
Figure 5.0: Command Prompt shows Previous running: None
- In this demo, we will use sheet1 to create images and servers, so we input 1.
Figure 5.1: CBR backup details in Sheet1
- After inputting the number of sheets, the batch file will continue with executing the main.py Python file to create Images and ECSs.
- This Python file will retrieve the backup ID from sheet1, and use the backup ID to create an image and ECS.
- Figure 5.2 below showed that the images and ECS are successfully created using row 3 (6d8055a8-4375-46e4-a7e3-e57ba865f122) and row 4 (444fca5a-8a91-4398-b6fb-df9ba305e3ca) backup ID.
Figure 5.2: Image and ECS successfully created
Figure 5.3: Image successfully created in Huawei Cloud console
Figure 5.4: ECS successfully created in Huawei Cloud console
- If the image and ECS are successfully created, the Image ID and ECS ID will be saved in resource_list.xlsx.
Figure 5.5: Image and Server sheets in resource_list.xlsx
- Figure 5.6 below showed that the image was failed to create using row 2 (00f0ff7a-b5b2-41fb-90d1-d2d348923c60) backup ID.
- The image creation might fail if:
- The backup ID is already registered as an image.
- The backup ID provided in Excel is invalid.
Figure 5.6: Image creation failed due to the backup ID already used to create an image.
- If the image creation fails, the vault ID and backup ID will be saved in the “Failed” sheet in resource_list.xlsx.
- If the image is successfully created but ECS creation failed, the vault ID, backup ID, and image ID will be saved in the “Failed” sheet in resource_list.xlsx.
Figure 5.7: Failed sheets in resource_list.xlsx.
Once the resources are no longer used, It’s a good idea to remove all the resources you created so Huawei Cloud doesn’t charge you for them.
- Inside the hwc_automation folder, find delete_images.bat and double click to run it.
- This batch file will execute the deleteImage.py Python File.
- deleteImage.py will read the images’ ID from the resourse_list.xlsx Excel file and use the ID to delete Images in Huawei Cloud.
- Run delete_servers.bat. This batch file will execute the deleteServer.py Python file.
- deleteServer.py will read the servers’ ID from the resourse_list.xlsx Excel file and use the ID to delete servers in Huawei Cloud.
By using the tool that I built above, customer successfully migrate CBR backup from Huawei Cloud Singapore Region to Malaysia CAE. All the objectives of the artifact have also been achieved. Below are the objectives and achievements of this artifact:
Objective | Achievement |
---|---|
To automate the process of creating a Full-ECS image from CBR and creating an ECS from the created image using Huawei Python SDK. | Creating image from CBR and provisioning ECS with image are successfully automated using Python SDK. Repetitious task is reduced from 134 times to 26 times. By default, the task of creating image from CBR and provisioning with image need to do 134 times, but with this program, the task can be done by batch, approximately two to seven ECS per batch. |
To automate the process of resource cleaning using Huawei Python SDK. | Resources cleaning such as delete image and delete ECS are successfully automated using Python SDK. The resources can be deleted with only one click. (run the deleteServer.bat and deleteImage.bat batch file) |
To develop an automated program to reduce time consumption. | The time required is reduced by 5 times, arround 81.6% faster compare to manual way. |