Start a Conversation

This post is more than 5 years old

Solved!

Go to Solution

2239

June 19th, 2017 09:00

VMAX AF - Massive latency trying to format an RDM

We have been building out a new VMware 6.5 environment, and in it, we've deployed some new servers for our patient medical records system. This system requires Physical Compatibility RDM's for it's data drives. I carved them up on our brand new VMAC AF (there's virtually nothing else on this box yet) presented them to the cluster, and the SysAdmin attached them tot he VM's the same way we've been doing it forever with our VMAX 10K, and our DMX before that.

He attached it, created a partition, and told it to quick format. He looked back at it about a half hour later and the 2TB drive was still formatting.  He tried a smaller drive (200GB) to one of the other servers, and got the same result. I pulled up the VMAX AF, and in the Storage Groups Dashboard it was showing 1 Critical under compliance. I checked performance and it was showing 2,400 ms write latency on this storage group. Meanwhile,everything else on the box is showing sub-ms latency.

I verified zoning was correct (every zone is 1 initiator to 1 target) I tried attaching it to a VM in a different cluster within the 6.5 Environment. Same result. I attached it to a VM in our old 5.5 environment, it formatted in the blink of an eye. So I figured it was a 6.5 thing. I presented a device from our VMAX 10K over to one of the VM's we were working with in 6.5, it formatted just fine.

So, out of curiosity, I took one of the original devices that was presented as an RDM, and instead, I turned it into a DataStore in 6.5. Went to one of the VM's we had been testing with, added a hard drive, and created a 5TB VMDK. It formatted instantly, and no latency issues on the VMAX AF.

Does anyone have any ideas for me? We'd really appreciate it!

Thank you in advance!

Bimmer

21 Posts

June 20th, 2017 13:00

Turns out that the issue is a VMware 6.5 bug in build 5310538). The current workaround is to downgrade ESXi servers to the build 4887370, or a registry edit in the Windows VM's that disables TRIM support until VMware releases a patch.

Just in case anyone is interested the RegEdit is "HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\FileSystem\DisableDeleteNotification" set the value to 1 and reboot.

21 Posts

June 20th, 2017 06:00

We finally left one of the RDM's alone long enough to finish formatting. It took almost 2 hours (2TB) and since then (last night) none of the devices in the SG have been touched. This is my current performance summary. 1,231 ms write latency with "0" Host I/O's happening on that SG.

RDM Latency.png

No Events found!

Top