VMware VMW_RR_PSP vSphere/ESX 6.x

So you want to change the ESX pathing select to your datastores from MRU to RR, but the docs say you have to reboot ALL your hosts?

Recently I raised a request for my IaaS provider to change our pathing selection from the default MRU to RR.  The tech/helpdesk person did some google searching on topic and got confused by the content, where the VM KB articles said to reboot and other articles were unclear, and so raised a support ticket with VMware and supposedly with HP to clarify.  The response I got back from my IaaS provider was that both HP/VMware came back and said must reboot.

So I called bullshit on that, in fact, that’s the cheap/lazy answer.

The way it works is that, you can create a policy to map VMW_SATP_ALUA to VMW_RR_PSP,  and it’s going to automatically apply it to any new devices being added – it only affects new storage sure, existing storage won’t change without a)  host reboot or b) manually setting the paths on each LUN. A reboot is just the brute-force approach to save clicks.

I fully expected VMware Support to comeback with the “reboot your computer” answer, I got that same answer for many issues over the years since 3.5  (mostly that was about all you could do tbh), I was a bit surprised by HP also stating this given their own HP 3PAR + VMware 6 Best Practice Guide gives both options – Reboot or manually set the paths …

This is how I have always done it since ESX 3.5

Now, if I had 10 or more ESX hosts, then yeah sure, let’s reboot to save clicks!

It’s not to say that rebooting isn’t a valid choice. If you make quite a few changes across a system, a reboot might be needed to weed out quirks. In my case, it was impractical to do that due to various reasons and also unnecessary to perform vmotions across 100 or so VMs over a few days for what could be done in a few minutes.

ESXi Guest e1000 tweaks

Windows

Refer to this KB article @ VMWare
Poor network performance on Windows 2008 Server virtual machine

Linux

    Disable TCP Segmenation Offload
    ethtool -K eth0 tso off
    ethtool -K eth1 tso off
    Increase Descriptors
    ethtool -G eth0 rx 4096 tx 4096
    ethtool -G eth1 rx 4096 tx 4096

    You can either add these to rc.local (or your distros equivalent) or my preference is to create a package (rpm/deb) so you can push/pull these to systems easily.

    You could also create /etc/modprobe.d/e1000.conf and add the below

    alias eth0 e1000
    options e1000 RxDescriptors=4096,4096 TxDescriptors=4096,4096

    and/or in /etc/network/interfaces:

    auto eth0
    iface eth0 inet dhcp
         up ethtool -G eth0 rx 4096 tx 4096
         up ethtool -K eth0 tso off

If you are still experiencing issues, you may just need to bite the bullet and use the vmxnet3 driver which has eliminated packet-loss/drops in the large majority of cases.

check_mk_agent and ESXi 4.1

Decided to add our Esx servers to our check_mk monitoring suite today, ran into two small issues.

1. check_mk_agent uses bash interpreter – ESXi (4.1) uses ash
2. afaict, check_mk_agent relies on xinitd, ESXi uses inetd.

Solution?

Download the check_mk_agent package rpm for esx/linux and extract the usr/bin/check_mk_agent script and usr/bin/waitmax

Edit the first line of check_mk_agent to be

#!/bin/sh

save and exit, then scp both files to /usr/bin on the ESXi server.

ssh ./{waitmax,check_mk_agent} root@esx-host:/usr/bin

Next, scp /etc/services and /etc/inetd.conf from the ESXi server and make the following changes.

./services
Add line:

check_mk 6556/tcp check_mk_agent   # check MK agent

./inetd.conf
Add line:

check_mk stream   tcp   nowait   root   /usr/bin/check_mk_agent check_mk_agent

Upload the files back to the ESXi host and your (almost) done!

This is the basic jist of how to get it working but there’s a far easier way to do this on many hosts. I for one automated the process by creating a payload package, a deployment script and setup ssh keys.

Of course, then next part is to actually make this stuff persistent.