Some thoughts on interrupt moderation

Most 1GbE or 10GbE Network Interface Cards (NICs) support a feature called
interrupt moderation or interrupt throttling, which coalesces interrupts from the NIC
to the host so that the host does not spend all its CPU cycles processing interrupts.
However, for latency-sensitive workloads, the time that the NIC delays the delivery
of an interrupt for a received packet or for a packet that has successfully been sent
on the wire, is time adding to the latency of the workload.
Most NICs also provide a mechanism, usually with the ethtool command, to
disable interrupt coalescing. VMware recommends to disable physical NIC interrupt
moderation on the VMware ESXi™ host as follows:

# ethtool -C vmnicX rx-usecs 0 rx-frames 1 rx-usecs-irq 0 rx-framesirq 0

Where vmnicX is the physical NIC as reported by the ESXi command:

# esxcli network nic list

You can verify that your settings have taken effect by issuing the command:

# ethtool -c vmnicX

Note that although disabling interrupt moderation on physical NICs is extremely
helpful in reducing latency for latency-sensitive virtual machines, it can lead to
some performance penalties for other virtual machines on the ESXi host, as well as
higher CPU utilization to deal with the higher rate of interrupts from the physical
NIC.
Disabling physical NIC interrupt moderation can also defeat the benefits of Large
Receive Offloads (LRO), because some physical NICs (such as Intel 10GbE NICs)
that support LRO in hardware automatically disable it when interrupt moderation is
disabled, and the ESXi implementation of software LRO has fewer packets to
coalesce into larger packets on every interrupt. LRO is an important offload for
driving high throughput for large message transfers at reduced CPU cost, so this
trade off should be considered carefully. Additional details are available in the
knowledge base article, Poor TCP performance can occur in Linux virtual machines
with LRO enabled: kb.vmware.com/kb/1027511.

If the ESX host is restarted the above configurations must be reapplied.