Key day two operations for an enterprise running Tanzu and/or Kubernetes clusters would typically include automated discovery, monitoring and troubleshooting of management and workload clusters. With the radical and exponential increase in the amount of traffic, interconnected flows and relationships between containerised workloads, the need of powerful and insightful monitoring tools is greater than ever.
vRealize suite products like vRealize Operations and vRealize Network Insight provide insightful and automated discovery modes for your Tanzu and upstream Kubernetes clusters. This allows you to monitor your containerised workloads and clusters from a single pane of glass alongside the rest of your infrastructure.
In this blog post I am going to walk you through monitoring Tanzu Kubernetes clusters using vROPs for Kubernetes management pack (previously known as management pack for containers monitoring). If you are interested in monitoring Tanzu and Kubernetes clusters using vRealize Network Insight then you can check my previous blog posts HERE and HERE.
Lab Inventory
For software versions I used the following:
- VMware ESXi 7.0U3c
- vCenter server version 7.0U3
- TrueNAS 12.0-U7 used to provision NFS datastores to ESXi hosts.
- VyOS 1.4 used as lab backbone router and DHCP server.
- Ubuntu 18.04 LTS as jumpbox server.
- Ubuntu 20.04.2 LTS as DNS and internet gateway.
- Windows Server 2012 R2 Datacenter as management host for UI access.
- vRealize Operations version 8.6.3 with management pack for Kubernetes 1.7
For virtual hosts and appliances sizing I used the following specs:
- 3 x virtualised ESXi hosts each with 8 vCPUs, 2 x NICs and 64 GB RAM.
- vCenter server appliance with 2 vCPU and 24 GB RAM.
Installing vROPs management pack for Kubernetes
vROPs management pack for Kubernetes allows vROPs to monitor your Tanzu/Kubernetes clusters regardless where they run. vRealize Operations Manager can monitor multiple Kubernetes solutions, whether it is VMware Tanzu Kubernetes Grid (TKG), RedHat OpenShift, or Kubernetes on Amazon Web Services EC2, Azure, or Google Virtual Machines.
Step 1: Download and add management pack to vROPs
Please note, if you are using vSphere with Kubernetes to run your containerised workloads then you do not need to use management pack for Kubernetes in vROPs, since workload management monitoring is natively supported and added in vROPs and added automatically when you add your vCenter server as data source.
To download the management pack, navigate to VMware Market place at https://marketplace.cloud.vmware.com/ and make sure that you have an account created so you can download management packs.
In the search field, search for “management pack for kubernetes 1.7” and then click on Download to download the pak file.
Login to vROPs using an admin privileged account, then from the left menu, click Data Sources > Integrations. Go to the Repository tab in the right pane and on the Repository tab, click Add.
Browse to the location where you have downloaded the pak file, choose it and then click on upload (this may take several minutes). Once the upload is successful you should see the below:
Read and accept the EULA, and click Next, when vRealize Operations Management Pack for Kubernetes is installed, click Finish.
Step 2: Preparing Kubernetes Cluster for monitoring
There are different methods by which vROPS management pack for Kubernetes can pull monitoring metrics and resource utilisation from Tanzu/K8s clusters, the one I will use in this blog post is the cadvisor open source agent. cAdvisor is an open-source agent that monitors resource usage and analyses the performance of containers. It collects statistics about the CPU, memory, file, and network usage for all containers.
Install cadvisor pods using cadvisor YAML
Login to your Tanzu cluster that you would like to monitor and paste the contents of the cadvisor YAML file from HERE in a file called cadvisor.yaml and create it on your Tanzu/k8s cluster using the command:
kubectl create -f cadvisor.yamlYou do not need to modify anything in that YAML file if your docker directories are in default locations and no specific port number changes are required.
cadvisor system pods will be created under kube-system namespace and should talk around 3 to 4 minutes to deploy and become running, check the status of the created pods using the command:
kubectl get pods -n kube-system
Wait till the above pods are in Running state then proceed with the following steps.
Adding Tanzu Kubernetes clusters in vROPs
After the cadvisor pods are initialised and in running state, copy and paste the below information in an external file as you will need them when adding your Tanzu or Kubernetes clusters in vROPs.
From your bootstrap machine (machine from which you manage your Tanzu or Kubernetes deployment) run the following command:
Cat ~/.kube/configFrom the output you need to get out the following info:
- Kubernetes Master API URL (by default is https://<k8s cluster mgmt node or cluster ip>:6443)
- certificate-authority-data
- client-certificate-data
- client-key-data
The certificate information need to be pasted into the credentials when adding the Tanzu Cluster in vROPs.
Navigate to your vROPs UI and under Data Sources > Integrations choose Kubernetes
Then fill in the required details to connect to your Tanzu Kubernetes cluster
Validate the connection to the newly added cluster and make sure it is successful
Click on ADD you should then see your Tanzu/Kubernetes cluster added as an integrated account in vROPs
Step 3: Monitoring added Tanzu Kubernetes clusters
Once the above steps are completed you should be able to see your Tanzu/Kubernetes clusters in vROPs and interact with them as you do with standard vSphere workloads. Below are some screenshots from my homelab after completing the above steps.
Final words
vROPS management pack for kubernetes onboards Tanzu and Kubernetes clusters seamlessly into vRealize Operations expanding vrops monitoring, troubleshooting and capacity planning capabilities to Tanzu and Kubernetes deployments deployments.
