The ESXi TN has 2 vmk:

vmk10 and vmk11 (normally, only vmk10 should be used for VTEP IP Address)

[root@sc2-hs2-b1537:~] esxcfg-vmknic -l
Interface Port Group/DVPort/Opaque Network IP Family IP Address Netmask Broadcast MAC Address MTU TSO MSS Enabled Type NetStack
vmk0 Management Network IPv4 10.173.62.41 255.255.255.0 10.173.62.255 24:6e:96:95:0a:18 1500 65535 true STATIC defaultTcpipStack
vmk0 Management Network IPv6 fe80::266e:96ff:fe95:a18 64 24:6e:96:95:0a:18 1500 65535 true STATIC, PREFERRED defaultTcpipStack
vmk10 10 IPv4 192.23.213.12 255.255.255.0 192.23.213.255 00:50:56:61:01:21 1600 65535 true STATIC vxlan
vmk10 10 IPv6 fe80::250:56ff:fe61:121 64 00:50:56:61:01:21 1600 65535 true STATIC, PREFERRED vxlan
vmk11 11 IPv4 192.23.213.13 255.255.255.0 192.23.213.255 00:50:56:6e:cd:07 1600 65535 true STATIC vxlan
vmk11 11 IPv6 fe80::250:56ff:fe6e:cd07 64 00:50:56:6e:cd:07 1600 65535 true STATIC, PREFERRED vxlan
vmk50 d82506f3-d3c9-45b2-85de-4ea03a506691 IPv4 169.254.1.1 255.255.0.0 169.254.255.255 00:50:56:60:e0:9c 1500 65535 true STATIC hyperbus
vmk50 d82506f3-d3c9-45b2-85de-4ea03a506691 IPv6 fe80::250:56ff:fe60:e09c 64 00:50:56:60:e0:9c 1500 65535 true STATIC, PREFERRED hyperbus

note vmk50 is expected so that's OK here

Need to have a separate t1 for ert, services, dynamic services, pas-infra, pks-infra

This allows flexibility of creating lbs

Add pks snat pool in sample parameter file
place in PKS section

in case we just want to install NSX-T for PKS, some NSX-T objects are not necessary like the LB
(in the default param file, there is this section which is not useful with PKS deployment:
nsx_t_lbr_spec: |
loadbalancers:

Sample entry for creating

)

is it possible to have a switch parameter to specify the deployment:
PKS or PAS or BOTH

in case of PKS, step to create the LB is not activated for instance.

1. When there are multi AZ you need multiple clusters prepared
2. When opsmanager and bosh are on the management cluster
   Sometimes the additional clusters may have different uplink profiles so need to add the ability to connect them to their respective profiles.
   Least priority is for multiple Compute vCenters

Getting this error when it tries to install the ovas for the 2.2 branch

/tmp/build/647f336c/nsx-t-gen-pipeline/functions/deploy_ova_using_govc.sh: line 170: govc: command not found
No JSON object could be decoded
Using VM options for ova upload
/tmp/build/647f336c/nsx-t-gen-pipeline/functions/deploy_ova_using_govc.sh: line 38: govc: command not found

govc is getting installed in the container but the $PATH is not getting updated with /root/go/bin and govc is not copied into /usr/bin as specified in the dockerfile so deploy_ova_using_govc.sh can't find it

Default parameter fro mem res is set to false. need to be true and require change to false to disable reservations:

Memory reservation is turned ON by default with the NSX-T OVAs.

This would mean a deployment of an edge or a mgr would reserve full memory

leading to memory constraints

Set following flag to true - would keep reservation ON, recommended for production setups.

Set following flag to false - would turn reservation OFF, recommended for POCs, smaller setups.

nsx_t_keep_reservation: false # for POCs, small setups

for now, the SNAT rule in the param file looks like this:

Sample entry for PKS PKS-Infra network

• t0_router: DefaultT0Router
  nat_type: snat
  source_network: 10.1.1.0/24 # PKS Infra network cidr
  translated_network: 23.23.23.1 # SNAT External Address for PKS networks
  rule_priority: 8001 # Lower priority

SNAT is dangerous when the destination network is too wide (i.e destination IP = ANY).

for PKS, we usually restrict the destination IP to a specific CIDR (we don't use ANY because it can break internal communications, from T1 to T1).

request is to be able to specify destination IP (or destination_network) in the above NAT section

Hi,

we are struggeling to find the Ansible-Tasks which create the T1-Routers. In the video (blogpost) there are Tasks shown (Create T1, Create Downlinks), but we cannot see this in the repository.

Please advise us how to go forward. Usecase is that we have existing T0-Routers and try to generate T1+x.

Name of nsx-mgr and nsx-controller appear to be hardcoded. Allow overriding of the default names.

Everything is fine until the step where OVA are deployed to vCenter.
I don't see anything happening on vCenter.
and there is no error message on the pipeline output (I made sure VM Network PG exist or VSS exist on the ESXi host - not sure if this is needed for this particular step)

thanks in advance

All these steps take a long time (10 to 20 min in my lab) before deploying the OVA on vCenter:

calculate new NSX Manager OVF File Checksum
calculate new NSX Controller OVF File Checksum
calculate new NSX Gw OVF File Checksum
delete old checksum in .mf file
add new checksum in .mf file
delete old ova files
zip OVFs to OVAs using ovftool
remove OVF Folders with all Files

is it possible to cache the result?
reason is because if 1 parameter is modified in the nsx-t-parans.yml (like nsx manaer vm name for instance), we need to wait for a long time before getting the result of the operation.

thanks in advance

Change the "compute vCenter" to "Compute manager" as it is called in the NSX UI. Thais will also support the addition of more "Compute managers"

When running the install-nsx-t job get the following output from log (Noting happens in vsphere):
TASK [deploy NSX Manager] ******************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************
changed: [localhost] => (item=nsx-manager)

TASK [Wait 4 minutes before starting to verify the deployment] *****************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************
Pausing for 240 seconds
(ctrl+C then 'C' = continue early, ctrl+C then 'A' = abort)
ok: [localhost]

TASK [Check if NSX Managers deployment has finished] ***************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************
failed: [localhost] (item={'_ansible_parsed': True, '_ansible_item_result': True, '_ansible_no_log': False, u'ansible_job_id': u'608769329668.1235', 'failed': False, u'started': 1, 'changed': True, 'item': u'nsx-manager', u'finished': 0, u'results_file': u'/root/.ansible_async/608769329668.1235', '_ansible_ignore_errors': None}) => {"ansible_job_id": "608769329668.1235", "attempts": 1, "changed": false, "cmd": "/root/.ansible/tmp/ansible-tmp-1529068294.9-145461286954642/deploy_ova.py", "data": "", "finished": 1, "item": {"ansible_job_id": "608769329668.1235", "changed": true, "failed": false, "finished": 0, "item": "nsx-manager", "results_file": "/root/.ansible_async/608769329668.1235", "started": 1}, "msg": "Traceback (most recent call last):\n File "/root/.ansible/tmp/ansible-tmp-1529068294.9-145461286954642/async_wrapper.py", line 150, in _run_module\n (filtered_outdata, json_warnings) = _filter_non_json_lines(outdata)\n File "/root/.ansible/tmp/ansible-tmp-1529068294.9-145461286954642/async_wrapper.py", line 91, in _filter_non_json_lines\n raise ValueError('No start of json char found')\nValueError: No start of json char found\n", "stderr": "Traceback (most recent call last):\n File "/tmp/ansible_NMDIln/ansible_module_deploy_ova.py", line 164, in \n main()\n File "/tmp/ansible_NMDIln/ansible_module_deploy_ova.py", line 146, in main\n if resource_pool is not None and resource_pool != '':\nNameError: global name 'resource_pool' is not defined\n", "stderr_lines": ["Traceback (most recent call last):", " File "/tmp/ansible_NMDIln/ansible_module_deploy_ova.py", line 164, in ", " main()", " File "/tmp/ansible_NMDIln/ansible_module_deploy_ova.py", line 146, in main", " if resource_pool is not None and resource_pool != '':", "NameError: global name 'resource_pool' is not defined"]}
to retry, use: --limit @/tmp/build/6fab756d/nsxt-ansible/deploy_mgr.retry

in the param file, we cannot specify tags for TZ (the overlay one)

this is needed for PKS

weird error message (because the ESXi was successfully prep'ed and added as TN):

TASK [Check for Fabric Node Status failed] *************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************
skipping: [localhost] => (item={u'body': u"{links : None, ip_addresses : ['10.173.62.41'], create_user : admin, create_time : 1526930680017, last_modified_time : 1526930680017, discovered_ip_addresses : [], id : e64416ea-b02b-4b40-af57-740a97e27aee, display_name : ESXi-1.mylab.com, last_modified_user : admin, schema : None, description : None, tags : None, self_ : None, protection : NOT_PROTECTED, system_owned : None, fqdn : , revision : 0, external_id : e64416ea-b02b-4b40-af57-740a97e27aee, resource_type : HostNode}", '_ansible_parsed': True, 'attempts': 1, '_ansible_item_result': True, '_ansible_no_log': False, u'ansible_job_id': u'806783813815.811', 'failed': False, u'changed': True, u'object_name': u'ESXi-1.mylab.com', 'item': {'_ansible_parsed': True, '_ansible_item_result': True, '_ansible_no_log': False, u'ansible_job_id': u'806783813815.811', 'item': u'ESXi-1.mylab.com', u'started': 1, 'changed': True, 'failed': False, u'finished': 0, u'results_file': u'/root/.ansible_async/806783813815.811', '_ansible_ignore_errors': None}, u'finished': 1, u'invocation': {u'module_args': {u'display_name': u'ESXi-1.mylab.com', u'nsx_manager': u'10.173.62.44', u'node_username': u'root', u'nsx_passwd': u'VALUE_SPECIFIED_IN_NO_LOG_PARAMETER', u'nsx_username': u'admin', u'os_version': u'6.5.0', u'state': u'present', u'thumbprint': u'VALUE_SPECIFIED_IN_NO_LOG_PARAMETER', u'os_type': u'ESXI', u'ip_address': u'10.173.62.41', u'node_passwd': u'VALUE_SPECIFIED_IN_NO_LOG_PARAMETER'}}, u'id': u'e64416ea-b02b-4b40-af57-740a97e27aee', '_ansible_ignore_errors': True})

for info, the whole job ran successfully

nsx-t-params.yml:

esxi_hosts_root_pwd: VMware1! # EDIT - Root password for the esxi hosts
esxi_hosts_config: |
esxi_hosts:

• name: esxi-host1.mylab.com
  ip: 10.173.13.2
  root_pwd: VMware1!
  nsx_t_esxi_vmnics: vmnic1 # vmnic1,vmnic2...

after the install-nsx-t job, I don't see the ESXi host (10.173.13.2) as Transport Node.
However, I can see the ESXi host Fabric -> Nodes -> Hosts.