How Apply To works in NSX DFW

Introduction

When working with the NSX Distributed Firewall (DFW), one feature that often goes unnoticed or misunderstood is ‘Apply To’. Despite its importance, it is frequently underestimated or even ignored. This is unfortunate, as ‘Apply To’ is a powerful feature that can significantly influence how firewall rules are applied within an NSX environment.

In many VMware training courses, ‘Apply To’ is either poorly explained or not mentioned at all. As a result, administrators and engineers might miss out on opportunities to optimize their firewall rule configurations. Misunderstanding or neglecting this feature can lead to overly complex rulesets or unexpected behavior in distributed environments.

In this post, I aim to demystify the ‘Apply To’ feature. I will explore its functionality, demonstrate its practical use cases, and analyze how it impacts the behavior of firewall rules. By the end, you should have a solid understanding of how and when to use ‘Apply To’ effectively in your NSX environment.

Questions to Address

To fully understand the ‘Apply To’ feature in the NSX Distributed Firewall, it is essential to examine its behavior under various scenarios. In this post, I will address the following questions:

1. What happens if we use the <Applied To> field on the policy and set the <Applied To> rule as DFW?
   How does this configuration affect the scope and enforcement of firewall rules?

2. What will happen if we use a Security Group in the <Applied To> field in the rule, while the <Applied To> field in the policy is set as DFW?
   What interactions or overlaps should be expected between these configurations?

3. What happens if we use security group1 in the policy <Applied To> field and security group2 in the rule <Applied To> field?
   How do these overlapping or conflicting settings impact the rule application?

4. Does the <Applied To> field in the rule or the <Applied To> field in the policy take precedence?
   When both are defined, which one ultimately dictates the scope of rule enforcement?

By exploring these scenarios, I aim to clarify the nuanced behavior of the ‘Apply To’ feature and provide actionable insights for optimizing your NSX DFW configurations.

Practical Demonstrations with CLI

In addition to exploring the theoretical aspects of the ‘Apply To’ feature, I will use practical CLI commands to demonstrate where the rules are actually enforced within the NSX infrastructure. This hands-on approach will help you visualize and verify how ‘Apply To’ configurations are realized in practice.

By combining both conceptual explanations and practical examples, you will gain a deeper understanding of how to effectively use the ‘Apply To’ feature in real-world scenarios.

Test Setup

For this blog, I am using NSX version 4.2.2.1 to demonstrate the behavior of the ‘Apply To’ feature in the Distributed Firewall. The goal is to keep the setup simple yet effective for understanding how different configurations influence the enforcement of firewall rules.

Environment Overview

The test environment consists of three lightweight Linux virtual machines running Alpine:

• Alpine01: 10.10.20.10
• Alpine02: 10.10.20.20
• Alpine03: 10.10.20.30 (Control VM)

All VMs reside on the same NSX segment, simplifying the network to focus on rule behavior and ensuring consistent connectivity between VMs. Notably, Alpine03 serves as a control VM and is not part of any NSX Security Group. This ensures it remains unaffected by specific ‘Apply To’ configurations, providing a baseline for comparison.

Security Groups

To organize and manage the test VMs, I have created the following NSX Security Groups:

1. dFG_all_Alpine
   This group includes Alpine01 and Alpine02. It represents all Alpine VMs involved in the main testing scenarios.

2. dFG_Alpine01
   A dedicated Security Group for Alpine01, enabling granular control over rules specific to this VM.

3. dFG_Alpine02
   A separate Security Group for Alpine02, allowing isolated configurations tailored to this VM.

Alpine03 does not belong to any Security Group, making it a neutral VM for verifying how rules or configurations behave when no specific policies are applied. This provides a clear reference point to validate the impact of ‘Apply To’ settings.

Why Alpine03 as a Control VM?

Having a control VM like Alpine03 ensures we have a clean baseline to compare against during the tests. By keeping it outside of any Security Group, we can confirm that any observed effects are solely due to the configurations applied to Alpine01 and Alpine02. This approach eliminates ambiguity and helps highlight the true behavior of the ‘Apply To’ feature.

Additional Tools for Validation

To analyze the behavior further, I will use CLI commands such as summarize-dvfilter and vsipioctl getrules, as well as the NSX Traceflow tool. This combination ensures an in-depth understanding of where and how rules are enforced, both at the hypervisor and the network level.

Naming Convention: A Recommended Best Practice

As a personal convention, I prefix all custom Security Groups with dFG, which stands for Distributed Firewall Group. This prefix helps me quickly identify and differentiate groups used for NSX Distributed Firewall purposes from other objects in the environment.

I strongly recommend adopting a consistent and meaningful naming scheme for your NSX environment. A clear structure not only improves day-to-day management but also prevents confusion in larger setups with potentially hundreds of objects. Whether you use prefixes like dFG or other naming conventions, the key is consistency.

Why Use a Single ESXi Host?

Running both Alpine VMs on a single ESXi host is intentional. It allows me to leverage the ESXi CLI to show where and how the Distributed Firewall rules are realized during the tests. This provides deeper insights into the inner workings of the NSX DFW, bridging the gap between configuration in the NSX Manager and actual enforcement at the hypervisor level.

This setup offers a practical foundation to explore and analyze the ‘Apply To’ feature in detail, combining theoretical explanations with real-world CLI examples.

Verification with CLI Tools

To complement the theoretical understanding of the ‘Apply To’ feature, I will use practical CLI tools during the tests to demonstrate where and how firewall rules are enforced. This includes inspecting the ESXi host to see how the Distributed Firewall implements the configurations. Below are the CLI commands and tools I will use:

1. summarize-dvfilter | grep -A16 <VMName>
   This command retrieves detailed information about the virtual NIC (vNIC) associated with a specific VM. By identifying the correct vNIC name (e.g., <nic-XXXXXXX-eth0-vmware-sfw.2>), we can pinpoint the interface where the firewall rules are applied.

2. vsipioctl getrules -f <name from the vNIC>
   Once the vNIC name is identified, this command provides the complete list of firewall rules applied to that specific VM. This is a powerful way to verify the exact rules enforced at the hypervisor level.

3. NSX Traceflow Tool
   In addition to ESXi CLI commands, the NSX Traceflow tool can be used to simulate and trace packet flow through the network. This tool helps visualize the path packets take and how firewall rules affect traffic between VMs.

Practical Workflow

During the tests, the workflow will involve the following steps:

1. Use the summarize-dvfilter command to locate the vNIC name for the VM being tested (e.g., Alpine01 or Alpine02).
2. Retrieve the enforced firewall rules using vsipioctl getrules with the vNIC name as input.
3. Validate the observed behavior using the NSX Traceflow tool to ensure that the rules are applied as expected and to simulate specific traffic flows for further analysis.

These tools provide a hands-on approach to understanding how the ‘Apply To’ feature works, bridging the gap between configuration and real-world enforcement.

Test 1: What Happens If We Use the <Applied To> Field on the Policy and Set the <Applied To> Rule as DFW?

To begin our exploration of the ‘Apply To’ feature, we start with a simple scenario. In this test, I create a policy that allows ICMP traffic from any source to the Security Group dFG_all_Alpine. The ‘Apply To’ configuration is as follows:

• The rule’s <Applied To> field is set to DFW (Distributed Firewall).
• The policy’s <Applied To> field is set to the Security Group dFG_all_Alpine.

Configuration Details

1. Policy:

   • Name: ICMP Allow Test
   • Source: Any
   • Destination: dFG_all_Alpine
   • Service: ICMP
   • Action: Allow
   • <Applied To>: dFG_all_Alpine

2. Rule:

   • <Applied To>: DFW

Expected Behavior

In this configuration, the policy’s <Applied To> field limits the scope of the policy to the members of the dFG_all_Alpine group.

The expected result is that the rule is only enforced for traffic destined to the dFG_all_Alpine group (as defined in the policy’s <Applied To>) even if the rule is set do DFW.

Test Results

Using the ESXi CLI, I collected the following details for the vNICs associated with each VM:

• Alpine01:

  • Port: 67108887
  • vNIC name: nic-533240-eth0-vmware-sfw.2

• Alpine02:

  • Port: 67108888
  • vNIC name: nic-533279-eth0-vmware-sfw.2

• Alpine03 (Control VM):

  • Port: 67108889
  • vNIC name: nic-544799-eth0-vmware-sfw.2

Firewall Rules Observed

1. Alpine01 (nic-533240-eth0-vmware-sfw.2) and Alpine02 (nic-533279-eth0-vmware-sfw.2):
   Both VMs have the following rules applied:

ruleset mainrs {
# PRE_FILTER rules
rule 10216 at 1 inout protocol tcp strict from any to addrset a34212cb-acb2-49b3-b74c-7683c0345a19 port 22 accept;
# FILTER (APP Category) rules
rule 10217 at 1 inout protocol icmp from any to addrset a34212cb-acb2-49b3-b74c-7683c0345a19 accept;
rule 10217 at 2 inout protocol ipv6-icmp from any to addrset a34212cb-acb2-49b3-b74c-7683c0345a19 accept;
rule 2 at 3 inout protocol any from any to any reject with log tag 'debug';
}

Key Observations:

ICMP traffic is explicitly allowed (rule 10217) for both IPv4 and IPv6 from any source to the address set associated with dFG_all_Alpine. All other traffic is rejected (rule 2), demonstrating that the rules are scoped correctly to the policy’s field. Alpine02 (nic-533279-eth0-vmware-sfw.2): The rules applied to Alpine02 are identical to those observed for Alpine01

2. Alpine03 (Control VM, nic-544799-eth0-vmware-sfw.2):

ruleset mainrs {
# FILTER (APP Category) rules
rule 2 at 1 inout protocol any from any to any reject with log tag 'debug';
}

Key Observations:

Only the default reject rule is applied. Since Alpine03 is not a member of dFG_all_Alpine, the policy does not apply to this VM, confirming that the field in the policy limits enforcement to the intended scope.

Conclusion for Test 1 The results confirm the expected behavior:

The policy’s field effectively limits the scope of enforcement to the Security Group dFG_all_Alpine. Although the rule’s field is set to DFW, rules are only applied to the VMs within the policy’s scope. The Control VM (Alpine03), which is outside the Security Group, does not have the ICMP allow rule applied, demonstrating the precision of the field.

Test 2: What Will Happen If We Use a Security Group in the <Applied To> Field in the Rule and the <Applied To> Field in the Policy Is Set as DFW?

In this scenario, I configure a policy to allow ICMP traffic from any source to the dFG_all_Alpine Security Group. However, this time I modify the <Applied To> field as follows:

• The policy’s <Applied To> field is set to DFW (Distributed Firewall).
• The rule’s <Applied To> field is set to the Security Group dFG_Alpine01.

Configuration Details

1. Policy:

   • Name: ICMP Allow Test 2
   • Source: Any
   • Destination: dFG_all_Alpine
   • Service: ICMP
   • Action: Allow
   • <Applied To>: DFW

2. Rule:

   • <Applied To>: dFG_Alpine01

Expected Behavior

This configuration introduces a more restrictive scope at the rule level:

• Since the rule’s <Applied To> field is set to dFG_Alpine01, it should only be enforced on the VMs in this group.
• The policy’s broader <Applied To> field (set to DFW) may lead you to assume that the policy and its rules are applied everywhere, but actual enforcement should be limited to dFG_Alpine01 due to the rule-level restriction.

Results

Firewall Rules Observed

1. Alpine01 (nic-533240-eth0-vmware-sfw.2):
   The following rules were observed:

ruleset mainrs {
# PRE_FILTER rules
rule 10216 at 1 inout protocol tcp strict from any to addrset a34212cb-acb2-49b3-b74c-7683c0345a19 port 22 accept;
# FILTER (APP Category) rules
rule 10217 at 1 inout protocol icmp from any to addrset a34212cb-acb2-49b3-b74c-7683c0345a19 accept;
rule 10217 at 2 inout protocol ipv6-icmp from any to addrset a34212cb-acb2-49b3-b74c-7683c0345a19 accept;
rule 2 at 3 inout protocol any from any to any reject with log tag 'debug';
}

Key Observations:

The ICMP allow rule is enforced as expected, scoped to Alpine01, which is part of dFG_Alpine01.

2. Alpine02 (nic-533279-eth0-vmware-sfw.2): The following rules were observed:

ruleset mainrs {
# FILTER (APP Category) rules
 rule 2 at 1 inout protocol any from any to any reject with log tag 'debug';
}

Key Observations:

The ICMP allow rule is not applied to Alpine02, as it is not part of dFG_Alpine01.

3. Alpine03 (Control VM, nic-544799-eth0-vmware-sfw.2): The following rules were observed:

ruleset mainrs {
 # FILTER (APP Category) rules
 rule 2 at 1 inout protocol any from any to any reject with log tag 'debug';
}

Key Observations:

As expected, no ICMP allow rule is applied since Alpine03 is not part of any relevant Security Group.

Test 3: What Happens If We Use Security <group1> in the Policy <Applied To> Field and <group2> in the Rule <Applied To> Field?

In this test, I explore the interaction between the <Applied To> fields at the policy and rule levels when they reference different Security Groups. The configuration is as follows:

• The policy’s <Applied To> field is set to dFG_Alpine01.
• The rule’s <Applied To> field is set to dFG_Alpine02.
• The rule allows ICMP traffic from any source to the dFG_all_Alpine Security Group.

Configuration Details

1. Policy:

   • Name: ICMP Allow Test 3
   • Source: Any
   • Destination: dFG_all_Alpine
   • Service: ICMP
   • Action: Allow
   • <Applied To>: dFG_Alpine01

2. Rule:

   • <Applied To>: dFG_Alpine02

Expected Behavior

In this configuration, the policy’s <Applied To> field restricts the scope of the policy to dFG_Alpine01, which includes only Alpine01. However, the rule’s <Applied To> field is set to dFG_Alpine02, which includes only Alpine02.

The expected behavior is:

1. The policy’s <Applied To> field should dictate the overall scope, meaning the rule will only be enforced for members of dFG_Alpine01.
2. Since the rule’s <Applied To> field is set to dFG_Alpine02, the rule will not be applied to Alpine02, as it is outside the policy’s scope.
3. ICMP traffic from any source to dFG_all_Alpine will only be allowed for VMs within dFG_Alpine01.

Test Results

Firewall Rules Observed

1. Alpine01 (nic-533240-eth0-vmware-sfw.2):

ruleset mainrs {
# PRE_FILTER rules
rule 10216 at 1 inout protocol tcp strict from any to addrset a34212cb-acb2-49b3-b74c-7683c0345a19 port 22 accept;
# FILTER (APP Category) rules
rule 10217 at 1 inout protocol icmp from any to addrset a34212cb-acb2-49b3-b74c-7683c0345a19 accept;
rule 10217 at 2 inout protocol ipv6-icmp from any to addrset a34212cb-acb2-49b3-b74c-7683c0345a19 accept;
rule 2 at 3 inout protocol any from any to any reject with log tag 'debug';
}
ruleset mainrs_L2 {
# FILTER rules
rule 1 at 1 inout ethertype any stateless from any to any accept;
}

Key Observations:

ICMP traffic (IPv4 and IPv6) is explicitly allowed (rule 10217) for Alpine01. This behavior aligns with the policy’s field, as Alpine01 is part of dFG_Alpine01.

2. Alpine02 (nic-533279-eth0-vmware-sfw.2):

ruleset mainrs {
# PRE_FILTER rules
rule 10216 at 1 inout protocol tcp strict from any to addrset a34212cb-acb2-49b3-b74c-7683c0345a19 port 22 accept;
# FILTER (APP Category) rules
rule 2 at 1 inout protocol any from any to any reject with log tag 'debug';
}

Key Observations: No ICMP allow rule is present for Alpine02, as it is not part of dFG_Alpine01, which is referenced in the policy’s field. All traffic is rejected at the main ruleset level (rule 2), confirming that the rule-level field (set to dFG_Alpine02) does not enforce the rule here.

3. Alpine03 (Control VM, nic-544799-eth0-vmware-sfw.2):

ruleset mainrs {
# FILTER (APP Category) rules
rule 2 at 1 inout protocol any from any to any reject with log tag 'debug';
}

Key Observations:

The only rule applied at the main ruleset level is a default reject rule (rule 2), which blocks all traffic. This confirms that no ICMP allow rule from the policy or rule is applied to Alpine03, as it is not part of dFG_Alpine01.

Answering Question 4: Does the Rule <Applied To> Field or the Policy <Applied To> Field Take Precedence?

Through our tests, we can confidently answer this question: The policy <Applied To> field takes precedence over the rule <Applied To> field.

Key Findings:

• The policy’s <Applied To> field defines the overall scope of enforcement. This means that if a VM or Security Group is excluded by the policy <Applied To> field, no rules from that policy will apply to it, regardless of the rule <Applied To> field.

• The rule’s <Applied To > field cannot further restrict or extend enforcement within the scope defined by the policy. It simply ignored.

Evidence:

• In Test 3, we demonstrated that even though the rule <Applied To> field was set to dFG_Alpine02, the rule was not applied to Alpine02 because the policy <Applied To> field limited enforcement to dFG_Alpine01.
• This behavior clearly shows that the policy <Applied To> field is the deciding factor in scoping firewall rule enforcement.

By understanding this precedence, administrators can better design their NSX firewall policies to avoid conflicts or unintended behavior.

Why Use <Applied To> in NSX?

The <Applied To> field in NSX Distributed Firewall is a powerful tool that enables administrators to optimize rule enforcement and improve overall performance in their environments. While it might seem optional at first glance, there are several key reasons to leverage this feature:

1. Optimizing Resource Usage

By default, Distributed Firewall rules are applied to all ESXi hosts in the cluster, even if they are irrelevant to some workloads. Using the <Applied To> field allows you to:

• Restrict rule enforcement to specific VMs, Security Groups, or segments.
• Reduce unnecessary rule propagation across unrelated hosts.
• Minimize the overhead of processing firewall rules.

2. Enhancing Rule Clarity and Management

When <Applied To> is used correctly, it provides clear boundaries for where rules are enforced:

• It helps avoid confusion about which VMs are affected by specific rules.
• It simplifies troubleshooting by narrowing down the scope of rule application.
• It prevents accidental rule application to unintended workloads.

3. Improving Security Posture

Restricting the scope of firewall rules reduces the attack surface:

• Only the VMs, segments, or groups explicitly defined in the <Applied To> field will be impacted by the rule.
• This minimizes the risk of unintentionally exposing unrelated workloads to less restrictive rules.

4. Avoiding Overlap and Rule Conflicts

In complex environments, rules can overlap or conflict, leading to unexpected behavior. By carefully defining <Applied To> fields:

• You ensure that rules are scoped to their intended targets, reducing the risk of conflicts.
• You can isolate specific rule sets for testing or special cases without affecting unrelated traffic.

Conclusion

While the <Applied To> field might add an extra layer of configuration, it plays a vital role in optimizing NSX Distributed Firewall performance, clarity, and security. By carefully designing and applying <Applied To> settings at both the policy and rule levels, administrators can achieve a more efficient, secure, and manageable firewall implementation. In addition, there is a limit to the number of firewall rules that can be applied to an ESX host and to a virtual NIC. The current limits may change with the NSX version and can be looked up in the Configmax tool from Broadcom.