Unsolved
This post is more than 5 years old
4 Posts
0
1272
June 30th, 2016 08:00
Does an SDC connect to every SDS or only one?
Does an SDC connect to only one SDS and that SDS connects to all other SDSs to retrieve, assemble, stream all the blocks to the SDC? Or does the SDC itself connect to every SDS to retrieve blocks and the SDC assembles the results received?
Related: Does the knowledge of where each block resides reside in the MDM or does every SDS (and maybe every SDC) know?
No Events found!
pawelw1
306 Posts
0
June 30th, 2016 11:00
Hi,
In general, whenever SDC comes up, it gets a map of chunks from the MDM, so it knows where (on each SDS) every chunk resides, so it can reach directly to the particular SDS in order to retrieve the blocks it needs - so answering your 2nd question, both MDMs and SDCs know the mapping. This information is stored in a very efficient manner, it takes only a couple of MBs of RAM to store mapping for a few PB of storage.
Whenever there is a change in mapping, the SDCs are not being updated immediately, but whenever the MDM decides to. Should the SDC need to retrieve the chunk it no longer resides on the particular SDS, ScaleIO error recovery mechanism will take care of that and it will be notified of the correct location without any disruption for the IO operations.
Hope that helps :-)
Br,
Pawel
pawelw1
306 Posts
0
June 30th, 2016 14:00
I can't think of any other solution than what you mentioned right now.. generally the volume is spread across the SPs so you probably could limit the SP to single rack, but that kind of defeats the purpose of ScaleIO.. or you can simply make sure there are enough links between the racks, so the traffic is not a problem.
meirlaker
4 Posts
0
June 30th, 2016 14:00
Thank you! Follow-up question to clarify: In a multi-rack system, with some nodes running ESX/SDC with VMs and other nodes running SDS with drives, is there any way to prevent cross-rack storage traffic of the VM to its storage? I think I would have to isolate all the VM's storage to the same rack as the VM by defining each rack as either its own Protection Domain or its own Storage Pool, neither of which is desired. (Currently, there is only one Domain for all racks. And, while there are multiple Pools, the Pools span drives across every SDS nodes in every rack.)
mpeters2
24 Posts
0
July 1st, 2016 05:00
Hopefully your nodes are connected via 10GbE networking (single or multiple links). I did the math on a dual 10GbE based node filling a full rack (20 nodes) that have 24 SAS drives in each node. Even with that density, the volume of traffic needed to force a network bottleneck was extremely high and not likely to be reached in any realistic general purpose scenario. So I agree with Pawel that as long as you have enough throughput between racks, you should not see any significant impact on traffic reaching nodes in different racks.