Code Monkey home page Code Monkey logo

hwcloud-apac-automation's Introduction

Automation on Elastic Cloud Server Provisioning using a created image from Cloud Backup and Recovery

1.0 Problem Statement

At the beginning, customer planned to use the existing server in Huawei Cloud Singapore region to perform data restoration from Cloud Backup and Recovery (CBR) and use Server Migration Service (SMS) to migrate restored Elastic Cloud Server (ECS) to CAE Malaysia. Unfortunately, all the existing server has been destroyed by customer and the proposed plan is not going to work anymore. The only solution for CBR migration is to create a Full ECS private image from CBR backup and use created image to provision a new ECS, then only perform migration using SMS to CAE.

However, the process of creating a Full-ECS image from CBR backup and creating ECS from the created images need to be done manually by customers using the Huawei Cloud console. Customers have to repeat the step from creating an image from CBR backup to creating ECS with the image created multiple times because there are 134 backups that need to be restored. This is very time-consuming and might have human errors.

1.1 Objective

In this artifact, we are going to achieve the below objectives:

  1. To automate the process of creating a Full-ECS image from Cloud Backup and Recovery (CBR) and creating an Elastic Cloud Server (ECS) from the created image using Huawei Python SDK.
  2. To automate the process of resource cleaning using Huawei Python SDK.
  3. To develop an automated program to reduce time consumption.

2. Solution Overview

Objective #1: Automate image creation from CBR and ECS provisioning

Solution Overview

Figure 2.0: image creation from CBR and ECS provisioning


Objective #2: Automate image and ECS cleaning

Solution Overview

Figure 2.1: image and ECS cleaning


3. Prerequisites

Before getting started, complete the following prerequisites:

  1. To use Huawei Cloud Python SDK, you must have a Huawei Cloud account as well as the Access Key and Secret Key of the Huawei Cloud account. You can create an Access Key in the Huawei Cloud console. You can obtain the 🔑 AK/SK pair for your account at My credentials > Access keys > Create Access key.

Create Credentials

Figure 3.0: Example of how to create Access Key


  1. To use Huawei Cloud Python SDK to access the APIs of a specific service, please make sure you have activated the service in the Huawei Cloud console if needed.
  2. Huawei Cloud Python SDK requires Python 3.3 or later, run the command python --version to check the version of Python. If Python 3.3 or later is installed. skip step 4.

Create Credentials

Figure 3.1: Command Prompt check Python version


  1. Install Python 3 and add Python to PATH. Open your browser and navigate to the Downloads for Windows section of the official Python website. Navigate to the Using Python on Windows section for steps to install Python in the Windows environment.
  2. Please note that this program is only applicable for Windows environments only for now.

4. Getting Started

Before starting to automate the process of creating images and Elastic Cloud Server (ECS) in Huawei Cloud, some steps need to be done:

  1. Download Source File
  2. Install Python Packages
  3. Configure Environment Variable

Download Source Code

  1. Click hwc_automation.zip to download the Source Code Zip archive file. Extract the Zip archive file after the download is complete.
  2. Check File Content.docx for the file content.

Install Python Packages

  1. Open the hwc_automation folder you have downloaded and extracted it in the previous step.
  2. Inside the folder, find install.bat and double-click to run it.
  3. This batch file will install all the Python Packages defined in requirement.txt.
  4. Python packages in requirement.txt
  • huaweicloudsdkecs
  • huaweicloudsdkims
  • huaweicloudsdksmn
  • huaweicloudsdksms
  • requests
  • python-dotenv
  • openpyxl
  • pandas

Configure Environment Variables

  1. Environment variables need to be declared first before proceeding to create Image, create ECS, delete Image, and Delete ECS.
  2. Inside the hwc_automation folder, find config.bat and double-click to run it.
  3. This batch file will start Command Prompt and ask the user to input parameters as prompted.
  4. The parameters required and steps to obtain as shown below:
Parameter Steps
Access Key & Secret Key (AK/SK pair) 1. On the console homepage, hover over the username in the upper right corner and choose My Credentials from the drop-down list.
2. Choose Access Key from the navigation tab at the left and click Create Access Key.
3. Click Download.
4. Open the credentials.csv you have downloaded to obtain AK/SK pair.
Admin Pass 1. On the console homepage, under Management & Governance, click Identity and Access Management.
2. In the navigation pane on the left, choose Account Security.
3. On the Account Security Settings page, select Password Policy in the navigation pane.
4. Follow the Password Policy to set Admin Pass. Admin Pass is required to create ECS and login into the ECS.
VPC ID 1. On the console homepage, under Network, click Virtual Private Cloud.
2. The VPC ID will appear once the cursor hovers over the VPC name.
3. After that, click the copy to clipboard icon beside the VPC ID to copy it.
Subnet ID 1. On the console homepage, under Network, click Virtual Private Cloud.
2. On the navigation pane at the left, click Subnets.
3. On the displayed Subnets page, locate your mouse cursor on the Subnets name.
4. The subnet ID will appear once the cursor hovers over the subnet name. After that, click the copy to clipboard icon beside the subnet ID to copy it.

Command Prompt

Figure 4.0: Command Prompt to input configuration parameters.


Obtain VPC ID

Figure 4.1: Example to obtain VPC ID


Obtain Subnet ID

Figure 4.2: Example to obtain subnet ID


  1. Parameter input by the user will save in the config.env environment variable file. You may open the hwc_automation > Code > config.env file to verify that the parameters are correctly declared.

5. Demonstration of Automation

Create Images and ECS

In this demo, we are going to automate the process of creating a Full-ECS image from Cloud Backup and Recovery (CBR) backup and creating an Elastic Cloud Server (ECS) from the created images. Step by step is as below:

  1. Inside the hwc_automation folder, find create_image_and_ecs.bat and double click to run it.
  2. This batch file will start a command prompt to show the number of sheets of the previous running and ask the user to input the number of sheets that need to be used in the current running as shown in Figure 5.0.

Note: None in the figure below indicate that the program never run before.

  1. The input is used to determine which sheet in ids_list.xlsx is to be used in creating images and servers. For example, if we input 1, sheet1 in ids_list.xlsx will be used while if we input 2, sheet2 will be used accordingly.

Command Prompt

Figure 5.0: Command Prompt shows Previous running: None


  1. In this demo, we will use sheet1 to create images and servers, so we input 1.

CBR backup details

Figure 5.1: CBR backup details in Sheet1


  1. After inputting the number of sheets, the batch file will continue with executing the main.py Python file to create Images and ECSs.
  2. This Python file will retrieve the backup ID from sheet1, and use the backup ID to create an image and ECS.
  3. Figure 5.2 below showed that the images and ECS are successfully created using row 3 (6d8055a8-4375-46e4-a7e3-e57ba865f122) and row 4 (444fca5a-8a91-4398-b6fb-df9ba305e3ca) backup ID.

Image and ECS successfully created Image and ECS successfully created

Figure 5.2: Image and ECS successfully created


IMG in HWC

Figure 5.3: Image successfully created in Huawei Cloud console


ECS in HWC

Figure 5.4: ECS successfully created in Huawei Cloud console


  1. If the image and ECS are successfully created, the Image ID and ECS ID will be saved in resource_list.xlsx.

ECS in HWC Image and Server sheets in resource_list.xlsx

Figure 5.5: Image and Server sheets in resource_list.xlsx


  1. Figure 5.6 below showed that the image was failed to create using row 2 (00f0ff7a-b5b2-41fb-90d1-d2d348923c60) backup ID.
  2. The image creation might fail if:
  • The backup ID is already registered as an image.
  • The backup ID provided in Excel is invalid.

image creation failed

Figure 5.6: Image creation failed due to the backup ID already used to create an image.


  1. If the image creation fails, the vault ID and backup ID will be saved in the “Failed” sheet in resource_list.xlsx.
  2. If the image is successfully created but ECS creation failed, the vault ID, backup ID, and image ID will be saved in the “Failed” sheet in resource_list.xlsx.

Failed sheets

Figure 5.7: Failed sheets in resource_list.xlsx.


6. Resource Cleaning

Once the resources are no longer used, It’s a good idea to remove all the resources you created so Huawei Cloud doesn’t charge you for them.

Delete Images

  1. Inside the hwc_automation folder, find delete_images.bat and double click to run it.
  2. This batch file will execute the deleteImage.py Python File.
  3. deleteImage.py will read the images’ ID from the resourse_list.xlsx Excel file and use the ID to delete Images in Huawei Cloud.

Delete Server (ECS)

  1. Run delete_servers.bat. This batch file will execute the deleteServer.py Python file.
  2. deleteServer.py will read the servers’ ID from the resourse_list.xlsx Excel file and use the ID to delete servers in Huawei Cloud.

7. Conclusion

By using the tool that I built above, customer successfully migrate CBR backup from Huawei Cloud Singapore Region to Malaysia CAE. All the objectives of the artifact have also been achieved. Below are the objectives and achievements of this artifact:

Objective Achievement
To automate the process of creating a Full-ECS image from CBR and creating an ECS from the created image using Huawei Python SDK. Creating image from CBR and provisioning ECS with image are successfully automated using Python SDK. Repetitious task is reduced from 134 times to 26 times. By default, the task of creating image from CBR and provisioning with image need to do 134 times, but with this program, the task can be done by batch, approximately two to seven ECS per batch.
To automate the process of resource cleaning using Huawei Python SDK. Resources cleaning such as delete image and delete ECS are successfully automated using Python SDK. The resources can be deleted with only one click. (run the deleteServer.bat and deleteImage.bat batch file)
To develop an automated program to reduce time consumption. The time required is reduced by 5 times, arround 81.6% faster compare to manual way.

hwcloud-apac-automation's People

Contributors

kekxian avatar cktanhuawei avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.