Code Monkey home page Code Monkey logo

nsxt-automated-lab-deployment's Introduction

NSX-T Automated Lab Deployment

Table of Contents

Description

Automated deployment of a fully functional NSX-T 2.x and vSphere 6.5 Update 1 environment which includes a set of Nested ESXi Virtual Appliance(s) configured w/vSAN as well as a vCenter Server Appliance (VCSA) using PowerCLI. For information, you can refer to this blog post here for more details.

Below is an screenshot of an example deployment:

Changelog

Requirements

Configuration

This section describes the credentials to your vCenter Server in which the NSX-T lab environment will be deployed to:

$VIServer = "vcenter.primp-industries.com"
$VIUsername = "primp"
$VIPassword = "MY-SUPER-DUPER-SECURE-PASSWORD-IS-VMWARE-123"

This section describes the location of the files required for deployment. All five variables are mandatory.

$NestedESXiApplianceOVA = "C:\Users\primp\Desktop\Nested_ESXi6.5_Appliance_Template_v1.ova"
$VCSAInstallerPath = "C:\Users\primp\Desktop\VMware-VCSA-all-6.5.0-5973321"
$NSXTManagerOVA = "C:\Users\primp\Desktop\nsx-unified-appliance-2.0.0.0.0.6522097.ova"
$NSXTControllerOVA = "C:\Users\primp\Desktop\nsx-controller-2.0.0.0.0.6522091.ova"
$NSXTEdgeOVA = "C:\Users\primp\Desktop\nsx-edge-2.0.0.0.0.6522113.ova"

This section defines the number of Nested ESXi VMs to deploy along with their associated IP Address(s). The names are merely the display name of the VMs when deployed. At a minimum, you should deploy at least three hosts, but you can always add additional hosts and the script will automatically take care of provisioning them correctly.

$NestedESXiHostnameToIPs = @{
"vesxi65-1" = "172.30.0.171"
"vesxi65-2" = "172.30.0.172"
"vesxi65-3" = "172.30.0.173"
}

This section describes the resources allocated to each of the Nested ESXi VM(s). Depending on the deployment type, you may need to increase the resources. For Memory and Disk configuration, the unit is in GB.

$NestedESXivCPU = "2"
$NestedESXivMEM = "12"
$NestedESXiCachingvDisk = "4"
$NestedESXiCapacityvDisk = "12"

This section describes the VCSA deployment configuration such as the VCSA deployment size, Networking & SSO configurations. If you have ever used the VCSA CLI Installer, these options should look familiar.

$VCSADeploymentSize = "tiny"
$VCSADisplayName = "vcenter65-1"
$VCSAIPAddress = "172.30.0.170"
$VCSAHostname = "vcenter65-1.primp-industries.com" #Change to IP if you don't have valid DNS
$VCSAPrefix = "24"
$VCSASSODomainName = "vsphere.local"
$VCSASSOSiteName = "virtuallyGhetto"
$VCSASSOPassword = "VMware1!"
$VCSARootPassword = "VMware1!"
$VCSASSHEnable = "true"

This section describes the location as well as the generic networking settings applied to BOTH the Nested ESXi VM and VCSA.

$VMCluster = "Primp-Cluster"
$VirtualSwitchType = "VDS" # VSS or VDS
$VMNetwork = "dv-access333-dev"
$VMDatastore = "himalaya-local-SATA-dc3500-1"
$VMNetmask = "255.255.255.0"
$VMGateway = "172.30.0.1"
$VMDNS = "172.30.0.100"
$VMNTP = "pool.ntp.org"
$VMPassword = "VMware1!"
$VMDomain = "primp-industries.com"
$VMSyslog = "172.30.0.170"
# Applicable to Nested ESXi only
$VMSSH = "true"
$VMVMFS = "false"

This section describes the configuration of the new vCenter Server from the deployed VCSA.

$NewVCDatacenterName = "Datacenter"
$NewVCVSANClusterName = "VSAN-Cluster"

This section describes the general NSX-T configurations and most of the defaults can be left alone outside of the envriomental specific configurations such as NSXPrivatePortgroup property.

$DeployNSX = 1
$NSXRootPassword = "VMware1!"
$NSXAdminUsername = "admin"
$NSXAdminPassword = "VMware1!"
$NSXAuditUsername = "audit"
$NSXAuditPassword = "VMware1!"
$NSXSSHEnable = "true"
$NSXEnableRootLogin = "true" # this is required to be true for now until we have NSX-T APIs for initial setup
$NSXPrivatePortgroup = "dv-private-network" # ensure this exists in your network

$TunnelEndpointName = "TEP-IP-Pool"
$TunnelEndpointDescription = "Tunnel Endpoint for Transport Nodes"
$TunnelEndpointIPRangeStart = "192.168.1.10"
$TunnelEndpointIPRangeEnd = "192.168.1.20"
$TunnelEndpointCIDR = "192.168.1.0/24"
$TunnelEndpointGateway = "192.168.1.1"

$OverlayTransportZoneName = "Overlay-TZ"
$VlanTransportZoneName = "VLAN-TZ"

$LogicalSwitchName = "Edge-Uplink"
$LogicalSwitchVlan = "0"

$ESXiUplinkProfileName = "ESXi-Uplink-Profile"
$ESXiUplinkProfilePolicy = "FAILOVER_ORDER" # Leave alone unless you know what you're doing
$ESXiUplinkProfileActivepNIC = "vmnic2" # vminic2 or vminic 3, Leave alone unless you know what you're doing
$ESXiUplinkProfileTransportVLAN = "0"
$ESXiUplinkProfileMTU = "1600"

$EdgeUplinkProfileName = "Edge-Uplink-Profile"
$EdgeUplinkProfilePolicy = "FAILOVER_ORDER"
$EdgeUplinkProfileActivepNIC = "fp-eth1"
$EdgeUplinkProfileTransportVLAN = "0"
$EdgeUplinkProfileMTU = "1600"

$EdgeClusterName = "Edge-Cluster-01"

This section describes the NSX-T Manager configuration. Currently, the re-sizing of the VM has not been enabled but will be possible in the future. For now, you can simply adjust the Hostname/IP Address.

$NSXTMgrDeploymentSize = "small"
$NSXTMgrvCPU = "2"
$NSXTMgrvMEM = "8"
$NSXTMgrDisplayName = "nsxt-mgr"
$NSXTMgrHostname = "nsxt-mgr.primp-industries.com"
$NSXTMgrIPAddress = "172.30.0.201"

This section describes the NSX-T Controller configuration. You can deploy N-number of Controllers by simply adding entries to the hash table. Currently, the re-sizing of the VM has not been enabled but will be possible in the future.

$NSXTCtrvCPU = "2"
$NSXTCtrvMEM = "6"
$NSXControllerSharedSecret = "s3cR3ctz"
$NSXTControllerHostnameToIPs = @{
"nsxt-ctr1" = "172.30.0.203"
"nsxt-ctr2" = "172.30.0.204"
"nsxt-ctr3" = "172.30.0.205"
}

This section describes the NSX-T Edge configuration. You can deploy N-number of Edges by simply adding entries to the hash table. Currently, the re-sizing of the VM has not been enabled but will be possible in the future.

$NSXTEdgevCPU = "2"
$NSXTEdgevMEM = "4"
$NSXTEdgeHostnameToIPs = @{
"nsxt-edge" = "172.30.0.202"
}

Once you have saved your changes, you can now run the PowerCLI script as you normally would.

Logging

There is additional verbose logging that outputs as a log file in your current working directory called nsxt20-lab-deployment.log which can be useful for debugging purposes.

Verification

Once you have saved all your changes, you can then run the script. You will be provided with a summary of what will be deployed and you can verify that everything is correct before attempting the deployment. Below is a screenshot on what this would look like:

Note: Although the resource requirements may seem quite high, I found in my lab setup, the memory footprint was signifcantly smaller, espeically for storage which ended up only consuming 67GB. For home labs, you can definitely reduce the memory of the NSX setup, but this was nothing something I had looked into great detail. The script will also have an option in the future to automatically shrink vCPU and vMEM based on user configuration but for the initial release, I stuck with the pre-defined defualts for what NSX-T calls a "small" deployment.

Sample Execution

Here is an example deployment:

If everything is succesful, you can now login to both your vCenter Server as well as NSX Manager. The NSX-T Manager has a new HTML5 UI which is decoupled from vCenter Server. To access it, open a browser to the following URL: https://[NSXT-MANAGER-ADDRESS] and using the admin credentials you had configured earlier.

Once logged in, you will see an overview of all your NSX Nodes by selecting the Dashboard* tab on the left side.

You can also verify that all your ESXi hosts have been properly configurd and prepared by selecting the Fabric tab on the left side and navigating to HOSTS tab at the top.

nsxt-automated-lab-deployment's People

Contributors

community-vsphere-tag avatar lamw avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

nsxt-automated-lab-deployment's Issues

Creating vSAN disk Group fails on reservation - Failed to reserve disk

during deployment. it happens on 2 Nested ESXi Hosts although fake scsi reservation is set during the script execution.

[01-02-2018_12:25:48] Adding ESXi host 192.168.1.172 to Cluster ...
[01-02-2018_12:25:55] Adding ESXi host 192.168.1.173 to Cluster ...
[01-02-2018_12:26:02] Enabling VSAN & disabling VSAN Health Check ...
[01-02-2018_12:26:30] Querying ESXi host disks to create VSAN Diskgroups ...
[01-02-2018_12:26:30] Creating VSAN DiskGroup for 192.168.1.171 ...
New-VsanDiskGroup : 1/2/2018 12:27:21 AM New-VsanDiskGroup Server task failed: A general system error occurred: Failed to reserve disk naa.6000c29214e6756d49a926a32ec4d244 with exception: Failed to reserve disk
naa.6000c29214e6756d49a926a32ec4d244 with exception: Reserve failed with error code: -1
At C:\Users\kay\Desktop\Nested_Deployment.ps1:898 char:13

  •         New-VsanDiskGroup -Server $vc -VMHost $vmhost -SsdCanonic ...
    
  •         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    
    • CategoryInfo : NotSpecified: (:) [New-VsanDiskGroup], VimException
    • FullyQualifiedErrorId : Storage_Impl_TaskResultConverter_TaskNotSucceeded,VMware.VimAutomation.Storage.Commands.Cmdlets.Vsan.NewVsanDiskGroup

Unable to add ESXi host(s) as a Transport Node

When executing the if($runAddESXiTransportNode) {} block, the following create method fails with an invalid_request message. The $transportNodeSpec variable output is provided below. It looks as though it may be missing some required values. Please advise.

PS C:\> $transportNode = $transportNodeService.create($transportNodeSpec)
A server error occurred: 'com.vmware.vapi.std.errors.invalid_request': . Check $Error[0].Exception.ServerError for more details.
At line:1 char:1
+ $transportNode = $transportNodeService.create($transportNodeSpec)
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo          : OperationStopped: (:) [], CisServerException
+ FullyQualifiedErrorId : VMware.VimAutomation.Cis.Core.Types.V1.CisServerException
PS C:\> $transportNodeService

Name Documentation
com.vmware.nsx.transport_nodes

PS C:\> $transportNodeSpec

self                     : <Unset, [com.vmware.nsx.model.self_resource_link]>
revision                 : <Unset, [long]>
description              : <Unset, [string]>
display_name             : ca-esx-1-123.corp.local
resource_type            : <Unset, [string]>
tags                     : <Unset, [List<com.vmware.nsx.model.tag>]>
host_switch_spec         : <Unset, [dynamic_structure]>
host_switches            : <Unset, [List<com.vmware.nsx.model.host_switch>]>
node_id                  : 55b057d6-459f-4c90-ac46-7f29b2b01296
transport_zone_endpoints : System.Object[]

Processing Certificate thumbprint failed - ObjectNotFound

[01-02-2018_12:28:46] Clearing default VSAN Health Check Alarms, not applicable in Nested ESXi env ...
[01-02-2018_12:28:46] Disconnecting from new VCSA ...
[01-02-2018_12:28:56] Successfully logged into NSX Manager nsxt-mgr.kay.local ...
[01-02-2018_12:28:56] Connecting back to Management vCenter Server vcenter.kay.local ...
[01-02-2018_12:28:57] Retrieving NSX Manager Thumbprint ...
[01-02-2018_12:28:58] Configuring NSX Controller nsxt-ctr1 as control-cluster master ...
[01-02-2018_12:28:58] Sending admin username ...
[01-02-2018_12:29:00] Sending admin password ...
[01-02-2018_12:29:05] Sending join management plane command ...
[01-02-2018_12:29:36] Sending shared secret command ...
[01-02-2018_12:29:41] Sending control cluster init command ...
[01-02-2018_12:30:06] Configuring additional NSX Controller nsxt-ctr2 ...
[01-02-2018_12:30:06] Sending admin username ...
[01-02-2018_12:30:08] Sending admin password ...
[01-02-2018_12:30:13] Sending join management plane command ...
[01-02-2018_12:30:43] Sending shared secret command ...
[01-02-2018_12:30:48] Sending exit command ...
[01-02-2018_12:30:59] Sending root username ...
[01-02-2018_12:31:01] Sending root password ...
[01-02-2018_12:31:11] Sending get control cluster cert ...
[01-02-2018_12:31:36] Processing certificate thumbprint ...
Copy-VMGuestFile : 1/2/2018 12:31:36 AM Copy-VMGuestFile The underlying connection was closed: An unexpected error occurred on a receive.
At C:\Users\kay\Desktop\Nested_Deployment.ps1:1035 char:17

  • ... Copy-VMGuestFile -vm (Get-VM -Name $nsxCtrName) -GuestToL ...
  •             ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    
    • CategoryInfo : NotSpecified: (:) [Copy-VMGuestFile], ViError
    • FullyQualifiedErrorId : Client20_VmGuestServiceImpl_DownloadFileFromGuest_DownloadError,VMware.VimAutomation.ViCore.Cmdlets.Commands.CopyVMGuestFile

Get-Content : Cannot find path 'C:\Users\kay\AppData\Local\Temp\controller-thumbprint' because it does not exist.
At C:\Users\kay\Desktop\Nested_Deployment.ps1:1036 char:48

  • ... Thumbprint = Get-Content -Path $ENV:TMP\controller-thumbprint | ? {$_ ...
  •              ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    
    • CategoryInfo : ObjectNotFound: (C:\Users\kay\Ap...ller-thumbprint:String) [Get-Content], ItemNotFoundException
    • FullyQualifiedErrorId : PathNotFound,Microsoft.PowerShell.Commands.GetContentCommand

Joining Controller using the thumbprint certificate.

some weird behavior in the code when joining different controllers to the NSX-T Manager. here is how the issue takes place.

1- first controller join is successful and the thumb-controller file in /tmp/thumb-controller

[01-06-2018_01:44:35] Successfully logged into NSX Manager nsxt-mgr.kay.local ...
[01-06-2018_01:44:35] Connecting back to Management vCenter Server vcenter.kay.local ...
[01-06-2018_01:44:37] Retrieving NSX Manager Thumbprint ...
[01-06-2018_01:44:37] Configuring NSX Controller nsxt-ctr1 as control-cluster master ...
[01-06-2018_01:44:37] Sending admin username ...
[01-06-2018_01:44:39] Sending admin password ...
[01-06-2018_01:44:44] Sending join management plane command ...
[01-06-2018_01:45:15] Sending shared secret command ...
[01-06-2018_01:45:20] Sending control cluster init command ...

2- the controller thus-print controller file gets deleted from ctr1. I did monitor the session and saw the file get deleted once join is successful.

3- the scripts tries to join the second controller. it creates the file thus-controller in the second node of the controller cluster.

4- now here is the issue starting. the variable $nsxControllerCertThumbprint returns a value of null which does not execute the copy file to the local system c:\temp\thubprint-controller.

5- the error shows as follows.

[01-06-2018_01:45:45] Configuring additional NSX Controller nsxt-ctr2 ...
[01-06-2018_01:45:45] Sending admin username ...
[01-06-2018_01:45:47] Sending admin password ...
[01-06-2018_01:45:52] Sending join management plane command ...
[01-06-2018_01:46:23] Sending shared secret command ...
[01-06-2018_01:46:28] Sending exit command ...
[01-06-2018_01:46:38] Sending root username ...
[01-06-2018_01:46:40] Sending root password ...
[01-06-2018_01:46:50] Sending get control cluster cert ...
[01-06-2018_01:46:50] Sleeping for 60 Seconds ...
[01-06-2018_01:47:50] i am dealing with
[01-06-2018_01:47:50] nsxt-ctr2
[01-06-2018_01:47:50] Processing certificate thumbprint ...
Copy-VMGuestFile : 1/6/2018 1:47:51 PM Copy-VMGuestFile The underlying connection was closed: An unexpected error occurred on a receive.
At C:\Users\Administrator\Desktop\nested.ps1:1046 char:17

  • ... Copy-VMGuestFile -vm (Get-VM -Name $nsxCtrName) -GuestToL ...
  •             ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    
    • CategoryInfo : NotSpecified: (:) [Copy-VMGuestFile], ViError
    • FullyQualifiedErrorId : Client20_VmGuestServiceImpl_DownloadFileFromGuest_DownloadError,VMware.VimAutomation.ViCore.Cmdlets.Commands.CopyVMGuestFile

Get-Content : Cannot find path 'C:\Temp\controller-thumbprint' because it does not exist.
At C:\Users\Administrator\Desktop\nested.ps1:1047 char:48

  • ... tThumbprint = Get-Content -Path C:\Temp\controller-thumbprint | ? {$_ ...
  •               ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    
    • CategoryInfo : ObjectNotFound: (C:\Temp\controller-thumbprint:String) [Get-Content], ItemNotFoundException
    • FullyQualifiedErrorId : PathNotFound,Microsoft.PowerShell.Commands.GetContentCommand

My-Logger : Cannot bind argument to parameter 'message' because it is an empty string.
At C:\Users\Administrator\Desktop\nested.ps1:1048 char:27

  •             My-Logger $nsxControllerCertThumbprint
    
  •                       ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    
    • CategoryInfo : InvalidData: (:) [My-Logger], ParameterBindingValidationException
    • FullyQualifiedErrorId : ParameterArgumentValidationErrorEmptyStringNotAllowed,My-Logger

6- the script tries to join the additional third controller and the thumbprint-controller file gets created but the system gets the same error as in step number 5 as the value in Null.

Failed Deployment of NSX-T Edge - Host did not have any virtual network defined

Hi,
I get the following error while trying to deploy the edge OVA

Import-VApp : 1/1/2018 11:26:43 PM Import-VApp Host did not have any virtual network defined.
At C:\Users\kay\Desktop\Nested_Deployment.ps1:769 char:23

  • ... sxedge_vm = Import-VApp -Source $NSXTEdgeOVA -OvfConfiguration $nsxEd ...
  •             ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    
    • CategoryInfo : InvalidOperation: (:) [Import-VApp], OvfNoHostNic
    • FullyQualifiedErrorId : Client20_VappServiceImpl_ImportVApp_CreateImportSpecError,VMware.VimAutomation.ViCore.Cmdlets.Commands.ImportVApp

does this indicated any issues in the initial configuration.

Note: never tried this on NSX-T 2.0, I am running the script on 2.1 version of NSX-T

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.