TL;DR - if you're using an Orange Pi 5 Plus or Raspberry Pi 4/5, you might want to stick with 1.15/7.0.
Apologies for the lack of updates. I've run into more issues trying to get VCF running on the embedded AMD EPYC build. With ESXi on ARM Fling 2.0 released, the base hypervisor has been upgraded to ESXi 8.0 Update 3b. I haven't had much luck with it as the release does not include the USB NIC community driver that the 7.0/1.15 release had. This limits us to exactly one USB NIC, that will only work at 100Mb/s. The onboard network adapter for the Raspberry Pi 5 is a different model than the 4, and as such is not compatible with the uether driver.
Tested BOMs:
Orange Pi 5 Plus
- ADT-LINK Riser PCIe X4 3.0 PCI-E 4X to M.2 NGFF NVMe M Key 2280 Riser Card
- Dell Broadcom 5719
- Intel PRO 1000
- Intel i210-T1 (generic brand ordered off of Amazon)
- Intel i225-V
- UEFI version 0.11.2 commit d668228 from the edk2-rk3588 project
Raspberry Pi 5
- GeeekPi P02 PCIe Slot for Raspberry Pi 5
- Same NICs as above
- UEFI version 0.3 commit c1ca184 from the rpi5-uefi project
With the Orange Pi 5 Plus, the issue first appeared to be related to MSI-X. In previous blog posts, I had to disable MSI interrupts to get NVMe drives to work. The behavior witnessed with Arm Fling 2.0 is that if MSI is disabled with a NIC, it simply disabled the device. In UEFI release 0.9.1, there were some messages indicating that MSIX interrupts were still being allocated:
VMK_PCI: 599: 0000:01:00.0: allocated 3 MSIX interrupts
These messages were also accompanied by TX and RX hangs. While the link light remained up, I could not pull a DHCP address on any card that I tried.
I'm not sure about everything that changed between 0.9.1 and 0.11.2, but when I attempted it on the latest version, there were no messages indicating MSIX interrupts, yet the behavior remained the same; link light, no DHCP, tx/rx hangs.
With the Broadcom 5719, the ntg3 module has an advanced parameter to enable legacy interrupts. I disabled MSI interrupts, enabled legacy with the driver, only for it to purple screen on boot. None of the Intel modules have this option. I've documented all of these issues on the Broadcom forum, in this topic.
With the Raspberry Pi 5, I ran into similar issues, with messaging indicating "Failed to allocate MSI interrupts". There has been significant development for Linux on RPi5, but the UEFI build has stagnated, as the primary contributors have shifted their focus to RK3588 based SBCs.
I believe the path forward will be with the Orange Pi 5 Plus. The only thing left to test is with an enterprise-grade i210 network adapter. I ordered a used Dell-based i210-T1 which should be here soon. This adapter has been proven to work with Ampere based servers, and the hope is that something with the device firmware will prevent the tx/rx hang issues. If it doesn't resolve the problem, I'll have to hurry up and wait on the USB NIC fling to be re-introduced to the ARM fling. For now, I'm going to stay on the 7.0 release.
