IPMI Upgrade through PXE

The Supermicro nodes have IPMI modules built in. These can be accessed using either the dedicated IPMI NIC, or by sharing the primary NIC with the host OS. We have chosen to use the shared option to reduce cabling.

Out of the box the machines came with a version of IPMI that didn't work for us so we had to update the IMPI firmware using the yafukcs utility.

The standard approach is to perform this via USB key but due to the number of nodes this is not practical. Therefore I created a freedos image containing the required firmware and configuration files that can be PXE booted to the nodes in one go.

This process saves around 4 days of manual labour. We still have to manually gather the MAC addresses of the new IPMI interfaces and add them to DHCP, but there are ways we can speed this process up using FDB in the switches.

Once we have our bootable image we add the required firmware update files

autoexec.bat

@echo off set PATH=.;\;\LOCALE
echo Using US-English keyboard with US-English codepage [437]
ipmicfg.exe -m
1.bat
2.bat
fdapm coldboot 

1.bat

yafukcs.exe -full X8DTT_~1.IMA 

Although it was quicker to update only the required modules using the -auto option we had better success with re-updating all the modules using the -full option.

2.bat

ipmicfg.exe -m
ipmicfg.exe -dhcp on
ipmicfg.exe -dhcp
ipmicfg.exe -r 

Once we run the update we configure IPMI with ipmicfg.exe to DHCP and very importantly perform a BMC cold reset. Then we add the image to the PXE config on our boot server (puppet in this case)

The relevant lines in the /var/lib/tftpboot/pxelinux.cfg/

LABEL Supermicro IPMI Firmware Update
KERNEL /sysresccd/isolinux/memdisk
APPEND initrd=/sysresccd/isolinux/floppy.img 

And copy the image (floppy.img) to /var/lib/tftpboot/sysresccd/isolinux/

We can now boot into the new image to ensure it works by selecting the new menu option from the PXE boot menu.