Deploying NSX NAPP on upstream (a.k.a native) Kubernetes – Part II

Overview

In the second part of this blog post I will be finalising my NSX Application Platform deployment on upstream Kubernetes. In Part 1 of this blog series I deployed a vanilla Kubernetes cluster from the ground up, deployed VMware Antrea as CNI, MetalLb as load balancer provider and vSphere storage plugin (CSI provider) which is needed by NAPP to provision dynamic persistent volumes on our Kubernetes cluster.

In this final post, I will be starting the deployment of NSX Application Platform (NAPP) and verifying the deployment components.

Lab Inventory

For software versions I used the following:

• • VMware ESXi 8.0 IA
  • vCenter server version 8.0 IA
  • VMware NSX-T 3.2.1.2 large form factor
  • TrueNAS 12.0-U7 used to provision NFS data stores to ESXi hosts.
  • VyOS 1.4 used as lab backbone router and DHCP server.
  • Ubuntu 20.04.2 LTS as DNS and internet gateway.
  • Windows Server 2012 R2 Datacenter as management host for UI access.

For virtual hosts and appliances sizing I used the following specs:

• • 3 x ESXi hosts each with 12 vCPUs, 2 x NICs and 128 GB RAM.
  • vCenter server appliance with 2 vCPU and 24 GB RAM.
  • 4 x Ubuntu 18.04 server VMs for upstream Kubernetes cluster

Deploying NSX Application Platform

Step 1: Create a Kubernetes Storage Class to utilise vSphere CSI

In order for NAPP deployment process to be able to provision PVs on our Kubernetes cluster, we need to create a k8s storage class (more about that in VMware Documentation) and define some parameters in it, in my setup I defined the below storage class on controller node and applied it using “kubectl apply -f”

kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: vsphere-csi-storageclass
  annotations:
    storageclass.kubernetes.io/is-default-class: "true"
provisioner: csi.vsphere.vmware.com
allowVolumeExpansion: "true"
parameters:
  storagepolicyname: "k8s-policy" 
  csi.storage.k8s.io/fstype: "ext4"

The k8s-policy is a storage policy created in the vCenter I connected to in part one of this blog post. Once the above storage class is applied to your kubernetes cluster, navigate to your NSX and start NAPP deployment.

Step 2: Start deploying NAPP from NSX UI

From NSX UI navigate to System > NSX Application Platform and then click on DEPLOY NSX APPLICATION PLATFORM

Choose the Paltform target version and if you use private Harbor registery and Helm repo then you need to type them as URLs below.

Fill in the kubernetes cluster configuration parameters below:

Make sure that all pre-checks are completed successfully (NTP error is not a deployment blocker).

Review the deployment parameters and then click Deploy if all okay.

The deployment process is a lengthy process and it took about an hour to 90 minutes to finish. Just note, that NAPP pods takes a lot of time to deploy and they exceed the default time out values configured in NAPP deployment process, so make sure that you keep on retrying the deployment if it fails (unless NAPP pods are failing for obvious errors).

If the process go through smoothly you should see NAPP deployed similar to below:

If you login to your kubernetes controller node and run the below command:

kubectl get pods -n nsxi-platform

you should see output similar to the below

Hope you have found this blog post useful.