Slow san performance for VM

Also, regarding Delayed ACK.  That typically shows up as high latency when the IO load is low.  That network feature is artificially holding on to the ACK which shows greater latency than what actually exists.  Since the IO has already been completed the sender is just waiting for the ACK to confirm it. 

Don 

Hello, 

 Hopefully the failed battery is on the passive controller?   If so, then it won't' impact performance of the active CM. But it needs to be replaced. 

 This page has many Tech Reports and documents for EQL arrays 

https://www.dell.com/support/article/en-us/sln312611/ps-series-technical-white-papers-and-videos?lang=en

This tech report covers the best practices for ESXi with Dell EQL storage. 

https://downloads.dell.com/manuals/common/ps-series-vmware-vsphere-bp-tr1091_en-us.pdf

 Especially with SQL and Exchange, sharing multiple VMDKs or RDMs on a single Virtual SCSI adapter is a huge performance limiter.  Each VM can have up to four Virtual SCSI adapters.  Spreading the IO load across these adapters improves IO performance on busy VMs.  Also using the Paravirtual SCSI adapter is often better as well.  (Requires VMware tools to be installed) 

 Also setting the proper MPIO IOs per path so both the iSCSI connections will be used efficiently.  By default only one connection is active until it fails.   Another common issue is Delayed ACK.  That needs to be DISABLED. 

 These are all covered in the Tech Report. 

 Re: Two iSCSI connections.  It's extremely rare that more than two ports are needed.  ESXi doesn't usually need hundreds of MB/sec throughput.  It needs exceedingly fast IO response time.   If you monitor your disk IO on your servers I suspect you will see that is true in your case.  If you have a three lane highway but only two cars they won't go any faster for having that third lane available. 

 Regards,

Don 

After shutting down the 4 VMs run's Windows server 2016 (2 VMs from the 4 run Sql server 2016) i noticed that the 5th VM run's windows server 2008 R2 and Sql Server 2008 works very well also the vcenter vm works well.

Also disk latency on both vms that still power on get low after the shutdown operation.

this is a screenshot for disk latency from my vm that run windows 2008 R2.

Thanks @dwilliam62  for details i will check the links.

Thanks @Origin3k , so i can switch to the second controller or it will be a mistake ?

Hello, 

 You can failover which might improve things  but only 6 drives in R6 is a serious limitation. 

How many  ESXi servers are connected to that array?   

 Problem is attached images don't show up immediately here. 

 So your latest ones for example are still masked out. 

 Regards,

Don

Maybe ....  and even than its possible risky.

Yes you can restart the Array to change the controller and the other one comes the active one.

But... for security the cache is mirrored to the seconday CM that in a event of a controller module failure the system can swap the roles without data loss. This mirroring cant work when the 2nd. CM is absent or have battery problems.

To test it you can swap the CM roles and IIRC there is a checkbox on the CM configuration how the Cache is used when only on CM is available. Iam not in front of my Groupmanager... so klick on every tab and take a look to the bottom of the screen.. i think 2 different options are there. The system default is that the Cache is disabled when there is only one CM.

If performance is increased you have to think about whats happens if you have a power loss or CM failure. You will get data corruption and thats not what you wanna have when using a VMFS.

Regards,
Joerg 

about the ISCSI connections, i have another datacenter the connections are exist on both EQL interfaces eth0 and eth1, but in this datacenter the connections are exist only on eth0 even the eth1 is up.

This is a printscreens from the datacenter where the VMs are slow.

Thank you @dwilliam62  @Origin3k  for the details.

Hello, 

 You are very welcome!   I am glad that I could help out. 

 FYI:  you can not use just any drives in EQL arrays.  They have to be specifically made for that array. Even a "dell" drive isn't going to work. I.e. one from a Dell server or other Dell array like ME/ME/SC.  It has to be a Dell/EQL drive.   Dell doesn't sell drives for out-of-warranty arrays so you'll need to find a third party to get them. Between that and setting up the ESXi nodes and replacing your CM it will likely resolve your performance problem.  

If you get the drives, stage them in the slots but not fully seated.  Once you have them all staged, push them in one at a time, one right after the other.  in case you were curious, you can't mix RAID types either. These new drives will also be RAID6. 

 Regards,

Don 

@dwilliam62 ,

if he force the use of write back instead of write trough by untick the cache rule he is at risk. But he can find out if the missing cache is the root cause.

@MMASLOUH 

if you buy EQL Disk from third party if helps if your EQL running a modern FW, because if the new disk contains old RAID information the GUI refuses them to add. In newer release this is possible.

Regards,
Joerg

Hello, 
  re: EQL drive.  That is true though the  clear label command has been available for many years.  Along with support in the GUI.  The 4100's lowest supported firmware should have that command. 

 What version of firmware is that array running? 

 Regards,

Don