Unsolved
This post is more than 5 years old
1 Rookie
•
25 Posts
0
4042
July 26th, 2019 07:00
High Latency On One Controller SCv3020
I'm setting up a new SCv3020 and am having a problem with high latency, but only on one of the controllers. This is running with a 3 host vSphere cluster and is the only attached storage, via iSCSI. There is a mixture of SSD and SAS drives but all the data is currently on the SSD drives.
So say I have volume 1 and volume 2 on the top controller, when cloning a VM to the same datastore I see high latency up to say 50ms and in some cases higher. Whilst with volume 3 and 4 on the bottom controller, performing the same operation the latency is sub 2ms.
If I swap the volumes between controllers, so 3 and 4 are on the top and 1 and 2 on the bottom - then the high latency remains on the top controller and the bottom controller is fine.
As far as I can tell everything is configured correctly and there are no warnings in Storage Center of vCenter.
Dell support thought it might be an issue with the cache on the top controller, so they advised us to pull it out for a few seconds so it would lose power and the cache would be cleared. We've done this but the issue remains.
When setting up the vSphere hosts there were some error messages when Storage Center was configuring everything - invalid Storage API request I think. But on running the configuration again it said they were already mapped and everything looked ok.
Anyone got any ideas?
jon1628
1 Rookie
•
25 Posts
0
July 29th, 2019 07:00
I've looked in to this some more....
I've run load tests from within VM's and the performance is fine, throughput is good and latency under 2ms. The only time the high latency seems to be happening is when I'm cloning a VM from vCenter. I believe this should be using the SAN offload (Full Copy) so it suggests that is working with one controller but not the other? Any ideas what could cause that?
When the issue happens the high latency only shows at the controller and storage profile views in SC charting. The actual disks being written to all show sub 2ms latency. So again, seems something is up at the controller level?
DELL-Sam L
Moderator
•
7.6K Posts
0
August 1st, 2019 13:00
Hello jon1628,
Seeing as the issue stays on controller1 no matter which volumes are on that controller it would be best to have support use secure console into that controller so that we can watch when the operation is happening to see what is going on. I don’t think that there is an issue with your configuration, but it never can hurt to double check to make sure all is set as in the deployment guide for your SCv3020. https://topics-cdn.dell.com/pdf/storage-scv3000_deployment-guide_en-us.pdf
One other thing I would check before contacting support is that you are following Dell EMC SC Series Best Practices with VMware vSphere 5.x-6.x https://downloads.dell.com/manuals/all-products/esuprt_solutions_int/esuprt_solutions_int_solutions_resources/general-solution-resources_white-papers8_en-us.pdf
Please let us know if you have any other questions.
jon1628
1 Rookie
•
25 Posts
0
August 14th, 2019 01:00
We went through the configuration with Dell Support, whilst there were some issues but resolving them didn't help with the latency issue. Whilst they could see the latency issue, they couldn't see what was causing it so we ended up swapping out the entire controller which resolved the issue.
chenchen
1 Rookie
•
31 Posts
0
March 14th, 2022 16:00
Same issues with me. Have you figured it out yet?
DELL-Joey C
Moderator
•
4K Posts
0
March 15th, 2022 02:00
Hi @chenchen,
It seems you have posted new topic for you issue, I've commented on it.
https://dell.to/3MMMNr7