Debian 8 Under Bhyve Networking Problem: Packets Over 230 Bytes Truncated

We’re a big fan of FreeBSD’s Bhyve hypervisor, including writing an open source lightweight management interface for it called BMT.

While setting up a Debian 8 guest VM we ran into a perplexing issue where networking was not working correctly.

After much troubleshooting we narrowed down the symptom to be that packets larger than 230 bytes (ie; ICMP payloads of 203 and greater plus overhead) were being truncated by, we suspect, Debian’s VirtIO networking driver:

root@debian-blank1:~# ping -M do -s 203 172.17.66.1
PING 172.17.66.1 (172.17.66.1) 203(231) bytes of data.
^C
--- 172.17.66.1 ping statistics ---
1 packets transmitted, 0 received, 100% packet loss, time 0ms

Packets smaller worked fine:

root@debian-blank1:~# ping -M do -s 202 172.17.66.1
PING 172.17.66.1 (172.17.66.1) 202(230) bytes of data.
210 bytes from 172.17.66.1: icmp_seq=1 ttl=64 time=0.320 ms

The solution was to switch Bhyve over to use the new Intel E1000 network card emulation driver instead of VirtIO. This is achieved by replacing the virtio-net with e10000 in the -s flag’s options list of the bhyve flags. If you’re using BMT simply set VM_NET_DRIVER="e1000" in the guest config.

After that no issues were reported. This seems specific to Debian 8 OSes, Debian 9 had no such problem even on the same guest instance in our testing.