Unsolved
This post is more than 5 years old
1 Rookie
•
96 Posts
0
4573
August 31st, 2018 02:00
SC4020 - 10Gb/s - What iSCSI bandwidth to expect ?
Trying to get the max out of our SC4020s, It seems that we can't get more that ~300MB/s sustained read throughput.
This is fair but a long way to the theoretical max of a single 10Gb/s link... and we have two..
What do you think ?
Is there something to unlock/tweak to get the max out of 10Gb iSCSI ?
(Our real-life operation which we would speed-up is a tar of a number of large files (VM disks) from a 5TB volume to tape library.)
For testing purposes to eliminate other throttling, I have created a 200GB volume, on tiers 1 (SSD Write Intensive). I'm simulating the "real-life" read operation by cp-ing to /dev/null or tar-ing to /dev/zero).
The iSCSI computeur client is Oracle Linux 7. Two iSCSI path (multipath - multibus, rr_min_io=1). MTU is, of course, 9000.
DELL-Bob Mi
2 Intern
•
230 Posts
0
August 31st, 2018 11:00
There are several items to look in to when you are trying to maximize your throughput. Are you following the recommended iSCSI settings in the Dell Storage Center SC4020 Storage System Deployment Guide
ftp://customer:Y3V2s-uH@ftp.compellent.com/DOCUMENTS/690-052-001.pdf
Is this a physical or Virtual server?
Have you read through the recommended settings for Oracle?
https://www.dell.com/support/article/us/en/19/sln312469/sc-series-technical-documents?lang=en#OR
How are you testing? It looks like you are copying files. This is a single threaded process, and is not an accurate test. IOmeter is a more accurate test. Also if you are writing to WI-SSD's do you have write cache disabled? When writing a to WI-SSD disks the writes can be slowed down when going through the cache card.
You can refer to the Dell Storage Center System Manager 6.6 Administrator’s Guide
ftp://customer:Y3V2s-uH@ftp.compellent.com/DOCUMENTS/STORAGE_CENTER/X1G22.pdf
Page 51: Change Volume Cache Settings
Cache settings can be set at the volume level, however cache settings that are set at the Storage Center level override individual volume cache settings.
NOTE: Because an SSD is itself a memory device, Dell recommends disabling write cache for all volumes using SSDs to maximize performance for most applications. Disabling read cache for volumes using SSDs is application-specific and may or may not improve performance. Dell recommends testing applications prior to disabling read cache.
lolix2
1 Rookie
•
96 Posts
0
September 3rd, 2018 00:00
Hi,
Thanks for your reply.
This is physical server (R730Xd) running Linux (All things Oracle are not database. This is Oracle *Linux* OS. let's say it's a Red-hat flavor.)
Yes we're following the recommended iSCSI architecture. The storage has been setup by a Dell engineer.
When doing tests using IOMeter, ATTO, etc.. people argues that they are not "real life" and when testing using real life cases, people argue that that they are not relevant...
So, here I am with that real-life case : large files reading.
I've played with Volume caching w/o much improvement. Note that when dealing with large file (300GB) caching is not relevant, IMO. When the cache is full you're ending up with the "real" performances and not the cache performances
Furthermore I'm not talking about writing, but reading. And to isolate the reads from the writes, I test by writing to /dev/null/ or /dev/zero..
When reading a file over a 1Gb network, I get the expected bandwidth and a single process saturate the link.
When reading a file over a 10Gb network, I don't get the expected bandwidth and I'd like to understand where is the bottleneck (storage array, CPU, PCI-express, switch ?..)
So, Is there someone out there who can read from a Compellent Volume at ~1GB/s, *sustained* ?