So, what’s happening? Granted this is an older Sun 4170M2 server, but  I have installed Oracle VM on the server with no issues. I also have run KVM hypervisors, Windows Hyper-V and Citrix without any issues. Well after a little troubleshooting it looks like VMWare does not like the video subsystem and is getting confused when it is checking to see if the server is headless.

The fix is relatively easy, we have to add a boot parameter to the ESXi kernel. To do this, reboot, and while ESXi is booting hit the tab key.

• Next, we will add a parameter to disable the headless check “ignoreHeadless=TRUE”
• And now reboot and install normally…
• Continue the installation normally, but when you get to the end pay attention.
• Now reboot, and get ready to hit “Shift O” this time
• The add in the ignoreHeadless=TRUE in the boot parameter list, it should look like this
• And then hit enter to continue the boot process.
• Next, once we are back up we will need to make the ignoreHeadless=TRUE permanent for each reboot.
• Log into the ESXi system as root, from the login screen hit F2
• Login as root
• Arrow down to troubleshooting options to enable the shell
• Next enable the ESXi shell
• The enter key will toggle the shell state.

Now from the console hit ALT-F2. On ILOMs you will need to use enable the ALT key from the keyboard menu.Then hit F2, to get to the shell login.

Don’t forget to uncheck the alt key from the keyboard dropdown once you are on the shell screen, otherwise you will have issues typing in the password!

next login as root, and we will run a single command

esxcfg-advcfg --set-kernel "TRUE" ignoreHeadless

I am a big fan of the concept of “trust, but verify”, so let’s verify that the parameter is correct by getting the parameter with the command.

“esxcfg-advcfg –get-kernel ignoreHeadless”

Now we  are done. type exit to logout, and then ALT-F2 to go back to the main screen.