How to get most out of your Nuc

First things first

Get a second NIC. The Intel NUC Pro has an IO expansion and supports an additional NIC. Unfortunately, these are relatively difficult to get in Germany, but it’s worth the effort.

vSphere 8 supports the cards natively and you don’t have to install any drivers. It also supports jumbo frames, which is relevant for NSX Labs. It is recommended to use a 2.5 GB managed switch. I am using a Mikrotik with the wonderful name CRS326-4C +20G+2Q.

Memory Tiering

Memory Tiering is very new in ESXi vSphere 8.0U3 and is still a Tech Preview. With memory tiering, you can use up to 400% of the physical RAM. This requires a fast NVMe. I would recommend a PCIe4 NVMe with at least 5000 MB/s read/write. Memory Tiering stores very cold (unused RAM pages) and cold RAM pages (less than 20% used) on the NVMe (Memory Tier). There is a wonderful Explore Session on this.

To enable memory tiering, you have to enter the following commands via the ESX Cli:

• This command turns on memory tiering

esxcli system settings kernel set -s MemoryTiering -v TRUE

• This command selects the NVMe

esxcli system tierdevice create -d /vmfs/devices/disks/<Your NVME>

• Enter the factor here (0-400%).

esxcli system settings advanced set -o /Mem/TierNvmePct -i 400

After a reboot, you have the selected amount of additional memory.

Using P/E Cores

Intel has introduced the big.little CPU architecture from the 12th generation of their consumer CPUs. This leads to some problems with ESXi. If the efficiency cores are activated, the ESXi starts with a PSOD (Purble Screen of Death). Fortunately, there are a few workarounds here.

• Disable the E cores in the BIOS

This means that you can use hyperthreading and the P Cores. However, you are clearly wasting potential here. That’s why we don’t want to.

• Use P and E cores and sacrifice hyperthreading for them

My tests showed that I got significantly more performance out of my 13th generation i7 if I didn’t use hyperthreading and only used “real” CPU cores, even if the E cores have a lower clock rate. William Lam has written very detailed blog articles about this, I link to him here for more information, as this article was actually only intended to be a short summary.

We actually only need two ESX CLI commands to make it all work.

• With this command, we prevent the PSOD from occurring when the ESXi boots.

esxcli system settings kernel set -s cpuUniformityHardCheckPanic -v FALSE

• With this command, we prevent the ESXi from getting a PSOD when the VMs are switched on.

esxcli system settings kernel set -s ignoreMsrFaults -v TRUE

If everything is correct, an ESX NUC of the 13th generation looks like this.