Securing Openshift Clusters Using VMware Antrea and NSX 4.1 – Part II

Overview

In part two of this blog post, we will be using NSX DFW to micro-segment containerised application running inside an openshift cluster based on OKD 4.12 which we previously integrated with NSX using VMware Antrea in part one of this blog post.

This is going to be fairly a short post since most of the work has been covered in the first part, in this post I will be covering the following points:

• Deploying Developers and Production Namespaces
• Create NSX Antrea Security Groups
• Configure NSX DFW micro-segmentation rules
• Verify micro-segmentation rules

Lab Inventory

For software versions I used the following:

• VMware ESXi 8.0U1
• vCenter server version 8.0U1a
• VMware NSX 4.1.0.2
• VMware Antrea 1.7.0
• VMware NSX ALB 22.1.3 for Openshift Cluster load balancing requirements.
• Redhat OKD 4.12
• TrueNAS 12.0-U7 as backend storage system.
• VyOS 1.3 used as lab backbone router, NTP and DHCP server.
• Ubuntu 20.04 LTS as Linux jumpbox.
• Windows Server 2019 R2 Standard as DNS server.
• Windows 10 Pro as UI jump box.

For virtual hosts and appliances sizing I used the following specs:

• 3 x virtualised ESXi hosts each with 12 vCPUs, 2x NICs and 128 GB RAM.
• vCenter server appliance with 2 vCPU and 24 GB RAM.
• NSX Manager medium appliance

Deploying Developers and Production Namespaces

In this step, I will create two namespaces Developers and Production in my openshift cluster, each with a cURL pod. The idea is to allow developers pods to access my microservices app which I deployed in part one of this blog post, while denying access to production namespace to that application.

Step 1: Deploy namespaces and cURL pods

From my jumpbox, I deployed the below namespaces and pods

apiVersion: v1
kind: Pod
metadata:
  name: curl
spec:
  containers:
  - name: curl
    image: curlimages/curl
    command: ["/bin/sh", "-c", "while true; do sleep 3600; done"]
    ports:
    - containerPort: 80

For testing I deployed simple cURL pods under both namespaces and for verifying micro-segmentation rules later on, I will initiate an HTTP session from both pods to a pod in my microservices app. For now, lets make sure that both pods can reach my microservices frontend service. Below is the services that are deployed in my microservices namespace:

The highlighted is a cluster service which maps to backed pods running my testing application homepage, I will curl to it from both pods under developers and production namespaces, we should receive a response

Create NSX Antrea NSX Security Groups

Before we can create and apply NSX DFW rules on our namespaces/pods we need to create three NSX Antrea security groups , from NSX UI navigate to Inventory > Groups > ADD GROUP and add 3 groups, one for developers with membership criteria namespace name developers:

Repeat the same for production namespace

For microservices, I am using membership criteria based on cluster service name, which is called frontend on port 80 (see output of oc get svc -n microservices in previous section) and maps to backed pod on port 8080 which is hosting my webpage for my application

Click APPLY, you then should see all the 3 groups created. Important note, if at this stage you try to view Antrea group members, they will be empty since we need first to create DFW rules for those groups in order to populate group members in inventory view.

Create NSX Antrea DFW Micro-Segmentation Rules

From NSX UI, navigate to Security > Distributed Firewall and then under Category Specific Rules, add a policy called Antrea-OCP and make sure that in the Applied To field to select your Antrea OCP cluster as shown below

Add 2 DFW rules as shown below to the newly create Antrea DFW policy, notice that I created a specific service on port 8080 since the HTTP traffic generated from the testing pods to frontend service will eventually be mapped to the endpoint pod which serves our webpage on port 8080, thus we need to filter on that port and not HTTP (TCP 80).

Once the DFW policy and rules are in place, click on PUBLISH to apply the DFW changes.

Verify NSX DFW Micro-Segmentation Rules

Switching back to our Openshift cluster, I will re-run the cURL commands from my developers cURL pod and another one from my production cURL pod:

As expected, the cURL from developers pod works while the same command from production pod does not, this is due to the NSX DFW rules we applied before.

We can also verify the network policy that NSX has applied to our Antrea agents by looking up the Antrea controller pod using the antctl command line tool from within Antrea controller pod

antcl is the command line tool for Antrea CNI and can be used to retrieve info about various Antrea featureGates, config parameters and networking policies. Below is the output of the two commands antctl get networkpolicy and antctl get networkpolicy <network_policy_name> 

The above is the YAML output of the network policy translation of our NSX DFW rules, further we can check the applied to groups as highlighted in the above screenshot, this can also be done using antctl command line tool from within controller pod

Hope you have found this post useful.