VCF 9 - NSX VPC Part 3 - Security

Introduction

In my two other VPC articles, I showed you the basics and the differences between centralized deployment and distributed deployment. Today, I would like to write something about security, as VPCs offer various possibilities in this area. VPCs provide a certain degree of isolation through their private networks, as these cannot be easily routed from outside the VPC. But what if we want to protect a VM from the public network or a private VM that has been assigned a public IP? This is where the distributed firewall and gateway firewall come into play. Let’s find out exactly how.

NSX Project

When it comes to isolation, we must also briefly address the topic of NSX Project. A Project is nothing more than a tenant in NSX. The name is debatable; personally, I don’t find it particularly apt.

There is always the default project, and most of my customers only use that and are happy with it. However, I don’t have any isolation (per default) through the distributed firewall in the default project. The default rule set initially allows all traffic, and in terms of VPCs, this means that everything that is routable can also be accessed.

But there is a solution. When creating a new project, you can have a default set of rules created that isolates the project from other tenants, including the default tenant. This can also be changed later. To do this, you must activate Activate Default Distributed Firewall Rules in the project settings.

The default Distributed Firewall rules allow communication between workloads within a Project, including communication with the DHCP server. All other communication is blocked.

The automatically generated rule set can be edited, expanded, and deactivated, but not deleted. A dynamic security group is automatically created, which includes all segments of the project. This security group creates an “allow any” rule within the project.

This completes the initial isolation between tenants, because even if the default tenant has an allow Any rule, the default drop rule of the project will reject the traffic if it comes from outside the tenant. However, this also means that my public VMs are no longer public, as they can only be accessed within the project. Internet access is also no longer possible, as all traffic to and from the project is blocked. I should change this.

Distributed Firewall

This is where the fun begins, and unfortunately we also have to take a look at the role model, because depending on which user role I am logged in with, I have different options for placing my distributed firewall rules. And the placement of the firewall rules is important for enforcing the rules or when they are enforced.

NSX VPC Role Model (Broadcom)

To better understand the table, it is important to know that rules can be created not only in the Security / Distributed Firewall tab, but also at the VPC level by going to the VPC settings and clicking E-W Firewall Rules for distributed firewall rules or N-S Firewall Rules for gateway firewall rules.

Rule order

It is important to note that if you create a firewall rule via the VPC and not via Security / Distributed Firewall, it will be created in the application category and will be applied according to the conventionally created Distributed Firewall rules, but will take precedence over the default Layer 3 policy. This means that the Project Admin or Enterprise Admin can override VPC rules at any time. By default, VPC rules are hidden in the Distributed Firewall. However, you can display the VPC objects in the DFW.

It is important to understand that the categories and apply to are still used to process the sequence. A deny all project rule in the Environment category takes precedence over a default project rule in the Application category with apply to dfw. However, the project rule is only valid within the project, even if apply to is set to dfw. A firewall rule from the default project with apply to set to dfw is enforced on all VMs. If rules from different sources are in the same firewall category, the default project rules are always processed first, then the project-specific rules, and only then the VPC rules – provided that the rules are relevant for the VM (apply to). If a default project rule is not enforced on the VPC VM via apply to, it is of course not processed. If no rule can be applied, our rule no. 2, also known as the default Layer 3 policy, comes into play. In production environments, this rule should always be a drop all and log rule.

To make this clearer, I have prepared an example. In my default project policy, I block http for all VMs (apply to dfw). In the project-specific policy, I block the https port for all project-specific VMs, and in the policy for VPC3, I allow icmp on all VPC3 VMs.

We can also view all of this on the CLI.

[root@vcf09-m01-esx01:~] vsipioctl getrules -f nic-1056617-eth0-vmware-sfw.2
ruleset mainrs {
  # generation number: 0
  # realization time : 2025-07-27T21:30:00
  # FILTER (APP Category) rules
  rule 12268 at 1 inout protocol tcp strict from any to any port 80 drop;
  rule 12264 at 2 inout protocol tcp strict from any to any port 443 drop;
  rule 11240 at 3 inout protocol icmp from any to any accept;
  rule 10216 at 4 inout protocol ipv6-icmp icmptype 136 from any to any accept;
  rule 10216 at 5 inout protocol ipv6-icmp icmptype 135 from any to any accept;
  rule 10217 at 6 inout protocol udp from any to any port {67, 68} accept;
  rule 10218 at 7 inout protocol udp from any to any port {546, 547} accept;
  rule 10219 at 8 inout protocol any from addrset 7a87cbca-ca80-4042-9507-558bb69b94c8 to addrset 7a87cbca-ca80-4042-9507-558bb69b94c8 accept;
  rule 10220 at 9 inout protocol any from any to any drop;
  rule 3 at 10 inout inet6 protocol ipv6-icmp icmptype 136 from any to any accept;
  rule 3 at 11 inout inet6 protocol ipv6-icmp icmptype 135 from any to any accept;
  rule 4 at 12 inout protocol udp from any to any port {67, 68} accept;
  rule 2 at 13 inout protocol any from any to any accept;
}

In the CLI output, you can clearly see the order in which the rules are processed. Here is the cross-check with a VM from VPC4. You can see that the default project rule and the project rule apply, but the VPC3-specific rule (ID 11240) does not exist.

[root@vcf09-m01-esx01:~] vsipioctl getrules -f nic-1056658-eth0-vmware-sfw.2
ruleset mainrs {
  # generation number: 0
  # realization time : 2025-07-27T21:36:42
  # FILTER (APP Category) rules
  rule 12268 at 1 inout protocol tcp strict from any to any port 80 drop;
  rule 12264 at 2 inout protocol tcp strict from any to any port 443 drop;
  rule 10216 at 3 inout protocol ipv6-icmp icmptype 136 from any to any accept;
  rule 10216 at 4 inout protocol ipv6-icmp icmptype 135 from any to any accept;
  rule 10217 at 5 inout protocol udp from any to any port {67, 68} accept;
  rule 10218 at 6 inout protocol udp from any to any port {546, 547} accept;
  rule 10219 at 7 inout protocol any from addrset 7a87cbca-ca80-4042-9507-558bb69b94c8 to addrset 7a87cbca-ca80-4042-9507-558bb69b94c8 accept;
  rule 10220 at 8 inout protocol any from any to any drop;
  rule 3 at 9 inout inet6 protocol ipv6-icmp icmptype 136 from any to any accept;
  rule 3 at 10 inout inet6 protocol ipv6-icmp icmptype 135 from any to any accept;
  rule 4 at 11 inout protocol udp from any to any port {67, 68} accept;
  rule 2 at 12 inout protocol any from any to any accept;
}

Gateway Firewall

To use the gateway firewall, it must first be activated for the project. The gateway firewall is implemented on the VPC gateway for VPCs and not on a T1 or T0 router. To activate the gateway firewall for a project, you must have at least project admin rights and can then activate it in the VPC Security Profile. It is then activated project-wide and can be configured by a VPC admin.

The rules are configured in the VPC setup under N-S Firewall Rules. They are not visible under Security / Gateway Firewall, regardless of the permissions my user has.

The configuration itself is very simple and straightforward. Of course, the gateway firewall does not work with the distributed transit gateway, as edge nodes are required for the gateway firewall. You also need to pay attention to the order in which they are processed. The gateway firewall only intervenes when traffic is sent via the VPC gateway to the transit gateway or vice versa. This means that internal VPC communication cannot be blocked.

For outgoing traffic, the distributed firewall is processed first, followed by the gateway firewall. For incoming traffic, the gateway firewall is processed first, followed by the distributed firewall. If the destination is located in another VPC or in another NSX project, the firewall is also processed at the destination as incoming traffic. If a gateway firewall is configured at T0, the order for outgoing traffic would be distributed firewall, gateway firewall on the VPC gateway, and then gateway firewall on T0, and vice versa for incoming traffic.

The gateway firewall does not offer us as many options as the distributed firewall, and it also requires edges. North-south traffic can also be effectively protected with the distributed firewall, although you may need to give a little more thought to the design of the rule set.

Bonus Round - TCP Strict

At the beginning of the article, I wrote that NSX has disabled TCP strict in the default policy. This is partly because TCP strict cannot work on an any/any rule; at least one TCP service must be specified. It is also irrelevant for a drop rule. But why is this the case? TCP Strict is a firewall mechanism that controls the 3-way handshake of TCP. In short, when the first packet is a SYN packet from the client, the firewall checks whether the subsequent packet is a SYN-ACK from the server. Only when the final ACK packet from the client arrives is the connection considered established and the firewall allows the user data to pass. Essentially, the firewall enforces clean TCP communication between the client and server and prevents half-open connections. Asymmetric routing is also blocked. This can cause problems with NSX in particular if you make the N/S peering of the edges to a firewall (this feature is called anti-spoofing in Checkpoint firewalls, for example).

If you now look at the default firewall rules of NSX for VPCs, you will quickly see that TCP Strict would not make sense in the policy.

This is because TCP Strict would not work with ICMP rules and the default DHCP rules (UDP) anyway, as there is no TCP traffic and TCP Strict does not apply with allow any/any or drop any/any.

No problem, right, Daniel? That’s exactly where my criticism comes in. Although you can’t delete the policy or enable TCP Strict, you can add as many firewall rules as you like – it’s so convenient and I don’t need to create a new policy. The only problem is that the new rules (part of the default policy) don’t have TCP Strict either, which weakens my firewall rules even though there’s no reason for it. The problem is that I can attack my server with modified TCP packets. This can happen intentionally or unintentionally (for example, due to a software bug). For example, I can easily bring down a test VM with TCP Ack packets using hping3. Normally, if TCP Strict is enabled, ACK packets without a preceding and matching SYN packet would simply be discarded by the firewall.

To perform a simple test, I added an Allow HTTPs rule to the default VPC firewall policy (this is the policy without TCP Strict) and bombarded my system with hping3 and TCP ACK packets.

hping3 -A --flood -p 443 -d 1200 10.28.11.35

Parameters:

• -A Sends TCP ACK packets (no connection establishment, simulates response traffic)
• –flood Sends packets as fast as possible without waiting for responses
• -p 443 Target port: 443 (HTTPS)
• -d 1200 Sets the payload size to 1200 bytes
• 10.28.11.35 Target IP address

The result is quite spectacular in BTOP. 100% CPU utilization of the VM and approx. 10 Gb/s network traffic, and everything passes through the firewall to the VM. To double-check, I create a new firewall policy. NSX always has TCP Strict enabled by default for new policies. The rule will be the same, allow HTTPs, but this time with TCP Strict enabled.

As you can see in the rule statistics, the new rules receive a large number of hits, but since there is no valid TCP handshake, the distributed firewall discards all packets and no packets arrive at the VM.

[root@vcf09-m01-esx03:~] vsipioctl getrules -f nic-1164294-eth0-vmware-sfw.2
ruleset mainrs {
  # generation number: 0
  # realization time : 2025-07-28T21:27:00
  # FILTER (APP Category) rules
  rule 13288 at 1 inout protocol tcp strict from any to any port 443 accept;
  rule 13289 at 2 inout protocol tcp from any to any port 443 accept;
  rule 5101 at 3 inout protocol ipv6-icmp icmptype 136 from any to any accept;
  rule 5101 at 4 inout protocol ipv6-icmp icmptype 135 from any to any accept;
  rule 5102 at 5 inout protocol udp from any to any port {67, 68} accept;
  rule 5103 at 6 inout protocol udp from any to any port {546, 547} accept;
  rule 5104 at 7 inout protocol any from addrset ba52d50f-55b3-453b-bff2-b2dd72beca3c to addrset ba52d50f-55b3-453b-bff2-b2dd72beca3c accept;
  rule 5105 at 8 inout protocol any from any to any accept;
  rule 3 at 9 inout inet6 protocol ipv6-icmp icmptype 136 from any to any accept;
  rule 3 at 10 inout inet6 protocol ipv6-icmp icmptype 135 from any to any accept;
  rule 4 at 11 inout protocol udp from any to any port {67, 68} accept;
  rule 2 at 12 inout protocol any from any to any accept;
}

In the CLI, you can also clearly see that rule 13288 uses TCP strict and rule 13289 does not.

What can we learn from this?

Firstly, you should think carefully about where and how you write your firewall rules, and secondly, you can use hping3 to disable your lab. :D

Furthermore, TCP Strict does not help with SYN flood attacks, as NSX creates a TCP session for each SYN packet and waits 120 seconds for the SYN-ACK. In the case of a hping3 attack, this would mean that the server would no longer be able to respond, especially since there is a parameter for random source and you end up simply occupying all of the server’s ports. Therefore, it may be useful to create a flood protection profile. These can be created separately for each project. However, you should be familiar with the traffic profile of the environment and, in general, caution is advised with such mechanisms, as overly strict settings can also block desired traffic.

Conclusion

But what can I do? My recommendation would always be to write your own policies for additional rules to the default VPC rule set. The VPC default rule set is very good for isolating the VPCs from each other, but that’s about it. Within the VPC, you should write your own policy, or at least one for traffic in and out of the VPC.

Use a perimeter firewall, as most attacks do not come from within, and learn how to write meaningful rules on the distributed firewall. Use Apply To, even on VPC rules, even if they are already restricted to the VPC. I have written about why Apply To is useful in another article. This also applies to the VPC topic.

Otherwise, all I can say is get yourself a KALI Linux and test your lab. I noticed the behavior with TCP Strict more by accident. It’s not a huge vulnerability, but you should be aware of what defaults and automatically generated rules or settings do. I don’t know yet if there will be another article on the topic of VPCs. My esteemed colleague Steven Schramm has taken a closer look at the topic of VPCs and VKS. It’s a very readable article.

So all that’s left for me to say is happy pen testing!