Booting the Orange Pi 5 Plus into ESXi on ARM

Disclaimer: Orange Pi products are not officially supported for ESXi on ARM. Even if you do get it to work, functionality will be limited at the time of this writing. I would not recommend purchasing an Orange Pi for the expressed purpose of ESXi on ARM. The folks working on the edk2-3588 UEFI project are doing amazing work, and I can't wait to see what comes next.

With that out of the way...

Booting and installing the ESXi on ARM Fling is possible on the Orange Pi 5 Plus. There are some pretty sizable caveats in doing so:

• On-board NICs are Realtek 8125, which do not have a compatible driver
• The only usable network adapter is the Realtek 8153 USB, which is capped at 100Mbps
• M.2 NVMe slot is not supported - having a device in the M.2 slot will cause drivers to hang when trying to boot
• Modification and creation of the edk2-rk3588 UEFI firmware project is necessary
• Hardware BOM needs to be used in specific ports in order to work
• Once installed, ESXi only recognizes the lone USB-C port, a hub is necessary
• As of this writing, the USB NIC Fling driver does not work with the ESXi on ARM Fling (no Fling-ception allowed)

This blog post will serve as a guide to get this installed, the bulk of which is modifying the firmware to work properly. My BOM is as follows:

• Orange Pi 5 Plus with 3D printed case and 40x40mm fan
• 16GB USB 3.0 thumb drive for ESXi installer (connected to the top USB 3.0 port)
• USB-C to USB A adapter
• Cable Matters 4 port USB hub, connected to the USB-C to USB A adapter 
• Cable Matters USB Network Adapter model 202013 (connected to hub)
• USB keyboard (connected to hub)
• USB 3.0 to SATA adapter with 1TB SSD as an install target/ESXi boot (connected to hub)
• 8GB Micro SD Card for edk2-3588 UEFI firmware (can use eMMC if your model comes with it, this guide will cover SD card)

The first step is to build out the UEFI firmware. The folks working on this project have been hard at work, and have successfully built firmware that allows things like the WoR Project (Windows on ARM) to gain some functionality. The base release works pretty well for Windows, but for ESXi, we'll have to build a custom version of it.

To do so, we'll need to use a Linux environment. I chose Ubuntu 23.04. The install instructions on the github page linked above include almost all of the packages needed. For my version of Ubuntu, I installed the following:

sudo apt install git gcc g++ build-essential gcc-aarch64-linux-gnu iasl python3-pyelftools uuid-dev device-tree-compiler

Then, clone the repository and change directory:

git clone https://github.com/edk2-porting/edk2-rk35xx.git --recursive

cd /edk2-rk35xx

Prior to building the firmware, we need to modify one of the files to disable PCI-e 3.0. If this step is not followed, the installation media will purple screen during boot. Use vim to modify this file:

sudo vi /edk2-rockchip/Platform/OrangePi/OrangePi5Plus/AcpiTables/AcpiTables.inf

Once you're in the file, comment the following line:

$(RK_COMMON_ACPI_DIR)/Pcie2x1l0.asl

Confirm that the line looks like this, then write changes:

# $(RK_COMMON_ACPI_DIR)/Pcie2x1l0.asl

Now we can build it (change release number to match the latest release found on the github page. As of this writing, 0.7.1 is the latest:

sudo ./build.sh --device orangepi-5plus --release 0.7.1

After a few minutes, the script should finish, and create a file named "RK3588_NOR_FLASH.img" in the edk2-rk35xx directory. You can then either use dd to copy the .img file onto an SD card, or SCP the file to a Windows machine and use the Raspberry Pi imager tool to write the file instead.

With the freshly imaged SD card inserted into the Orange Pi, we can now plug everything else into the machine. Remember, the device order is as follows:

• USB drive containing ESXi installer in the top USB 3.0 port
• USB hub plugged into the USB-C port
• Keyboard, SSD and Realtek 8153 adapter plugged into the hub

If all goes well, ESXi should boot without issue, and will see the network adapter and SSD. Install ESXi as you normally would. I made the mistake of thinking that I could put the USB SSD on the 3.0 ports after installing - this will work, meaning ESXi will still boot, but any additional capacity used for the datastore will be inaccessible, and ESXi will be read only. You must keep the SSD and network adapter on the hub. 

The promise of an 8 core, 16GB RAM, ARM based ESXi machine looks great. I'm hoping that in the not too distant future, we can see more improvements, such as additional USB port and NVMe disk utilization. A workaround for the lack of networking would be to pass another USB network adapter to a VM, which could act as a DHCP server and present gigabit connectivity for other guests, but this would only work if the other USB ports were recognized. 

Time will tell what the future holds for the Orange Pi 5 Plus, but to have UEFI functioning at this level is fantastic, and I can't say enough good things about the people working on this project. Also, a special thanks to the ESXi on Arm team, who have made this endeavor possible.