As usual nowadays, Lenovo SR650 servers come with energy saving presets that may seem ”green”, but which kill virtualization performance.
The regular way to get them running the way they should is to enter the UEFI setup at boot, go to UEFI Settings -> System Settings -> Operating Modes and choose ”Maximum Performance”. Unfortunately, on these servers, this removes the ability to set VMware EVC: the Enhanced vMotion Compatibility functionality that allows for live migration of virtual servers between hosts of different generations, for example when introducing a new cluster into a datacenter.
It turns out that what’s missing is one specific setting: ”MONITOR/MWAIT” must be set to ”Enabled”. It should be possible to first choose the ”Maximum Performance” scheme, then switch to the ”Custom” scheme and only change this single setting in Operating modes. In addition, we should also go to System Settings -> Devices and I/O Ports, and modify PCI 64-bit Resource Allocation to read ”Disabled”. For reference, the complete checklist is available from Lenovo:
Processors.CStates=Disable
Processors.C1EnhancedMode=Disable
Processors.EnergyEfficientTurbo=Disable
Processors.MONITORMWAIT=Enable
Power.PowerPerformanceBias=Platform Controlled
Power.PlatformControlledType=Maximum Performance
DevicesandIOPorts.PCI64BitResourceAllocation=Disable
DevicesandIOPorts.MMConfigBase=3GB
After making these changes, we should be able to both run our workload at maximum performance and enable EVC to migrate workloads between server clusters utilizing CPUs from different generations.