With the release of NSX 188.8.131.52 and VMware Antrea 1.5.0 came a very interesting announcement that you can now secure Antrea containers integrated with NSX using NSX IDPS i.e. leveraging threat prevention sigantures and capabilities to secure your containerised workloads.
This feature is however is tech preview only at the moment, which means it is not for production deployment and VMware does not offer any support for this at the moment. The feature also offers only IDS capability at the moment with no GUI integration with NSX IDPS alerting dashboards i.e. you need to either read the IDS events alerts through pod logging or by exporting pod logging to logging tool such as fluent (I will discuss this in later posts.)
VMware Antrea IDPS implementation is based on project Antrea’s Traffic Control capability that was introduced with Antrea 1.7.0 release and uses Suricata (open source IDS/IPS project) as the IDPS engine.
Vmware Antrea IDPS solution is composed of the following main components:
For software versions I used the following:
For virtual hosts and appliances sizing I used the following specs:
Traffic Control feature gate has been introduced as Beta in project Antrea 1.7.0. Traffic Control is a CRD API that manages and manipulates the transmission of Pod traffic. It allows users to mirror or redirect specific traffic originating from specific Pods or destined for specific Pods to a local network device or a remote destination via a tunnel of various types (ex. vxlan or geneve). It provides full visibility into network traffic, including both north-south and east-west traffic.
To enable Traffic Control on your Antrea agents, connect to your Kubernetes cluster (or from your master node) run the following command to determine the antrea configmap that you need to manipulate:
Now, edit the highlighted configmap using the following command:
kubectl edit configmaps -n kube-system antrea-config
This will open a vi editor for the antrea config agent, search for “TrafficControl” uncomment the line and set the value to true
save and exit vim using :wq! and then perform a rolling restart for Antrea config daemonsets to that changes in configmaps take effect:
kubectl rollout restart ds/antrea-agent -n kube-system
for this step, we need to use a deployment yaml file to roll out the IDPS controller and agents, you can use a sample deployment file I uploaded for you HERE just copy and paste the contents of this file to a file called idps.yaml (name does not matter) open it with any editor and add your NSX TP or ATP in base64 encoded format to the below section in that deployment file:
Save and exit the file and then apply it using the following command:
kubectl apply -f idps.yml
This will start creating Antrea IDPS controller and agents pods under kube-system system namespace, give it sometime and then check the status of pods using the command:
kubectl get pods -n kube-system
The status of all the pods must be running.
Once Antrea IDPS is deployed, we need to define an IDPS policy which basically tells the IDPS engine which traffic from which pods should be monitored and which engine to use, either an IDS our IPS. However, in the current tech review release of VMware Antrea, VMware only supports IDS mode, so no need to define the engine type in the the IDPS policy CRD that we are going to create.
Create a file called WebIdsPolicy.yaml and paste in the following contents:
apiVersion: crd.antrea.tanzu.vmware.com/v1alpha1 kind: IDPSPolicy metadata: name: test-ids-policy spec: appliedTo: podSelector: matchLabels: app: web
The above will create a test IDPS policy called test-ids-policy which is active on pods having label of app=web. Save and exit the above file (please make sure that your indenting is exactly as shown above).
Run the command “Kubectl apply -f WebIdsPolicy.yaml” to start creating the test IDS policy CRD.
Check that the CRD has been created successfully:
In order to verify that our Antrea IDS is working by detecting and reporting malicious traffic patterns and behaviour, we will create a pod running alpine-curl image (or any container image with curl in it) and will label it with app=web so that our previously created IDS policy get applied to this test pod.
One important note, in this test I will be testing an egress malicious attack from my curl pod to http://testmynids.org, this is a very cool website which allows security admins test their IDPS engine against this website. A list of the available attacks and how to launch them can be found on their Github repository as well:
For this test, I am going to use the most simple one which is a uid/gid scan against their website, so from my curl pod I will run the command:
curl -s http://testmynids.org/uid/index.html
For this I will also need to add my DNS to my Pod configuration, because by default pods will only use default cluster local dns to resolve only local kubernetes services and pod names. To create our testing pod, just copy and paste the following in yaml file and apply it:
apiVersion: v1 kind: Pod metadata: name: curl labels: app: web spec: containers: - name: source image: quay.io/cilium/alpine-curl command: ["sh", "-c", "while true; do sleep 3600; done"] dnsPolicy: "None" dnsConfig: nameservers: - 192.168.0.253
once you apply the above file using the command kubectl apply -f <pod yaml filename> get on which node this test pod is running using the command:
kubectl get pods -o wide
login to the above shown node and run the following command:
ls -l /var/log/antrea/suricata
Record the name of the created log file (this is the log file Suricata will be logging events to) then tail this file:
tail -f tail -f /var/log/antrea/suricata/<your eve filename>.json
Now, switch back to your bootstrap machine or your Kubernetes master and generate an egress malicious scan from our testing pod using the following command:
kubectl exec curl -- curl -s http://testmynids.org/uid/index.html
Switch back to the node on which the test pod is running and you should see the following event generated:
If you want to get a more neat output then you can pipe the above command to jq (on Ubuntu you can install it using sudo apt install -y jq)
With the addition of IDPS capability to Antrea, this will leverage the use of NSX drastically within the modern applications security space. Although the feature is in tech preview it is very promising and I expect this to be a game changer in NSX footprint across VMware customers.
Stay tuned to my next blog post in which I will show you how to visualise Antrea IDPS alerts for a more real life use case.[/vc_column_text]
Overview Note: After I have written this blog post I got to hear that work…
Overview I am just back from VMware Explore in Barcelona after presenting an interesting session…
Overview In the second part of this blog post I will be finalising my NSX…
Overview In a previous blog post (HERE) I deployed NSX Application Platform on top of…
Overview With the release of vSphere 8, VMware introduced Tanzu Kubernetes Grid clusters version 2,…
Overview vSphere 8 introduced zonal supervisor cluster deployments in order to improve Tanzu workload resiliency,…