Unsolved
This post is more than 5 years old
16 Posts
0
903
December 4th, 2016 20:00
On the specific difference of the rebuild and rebalance modes.
Hi
I would like to make sure of specific differences among the rebuild and rebalance modes.
What's confusing me is the following descriptions.
On one hand, predefined limits for Favor Application I/O and Dynamic Bandwidth Throttling are imposed regardless of Application I/O occurrence.
P40
"Favor Application I/O: .....The rebuild I/Os are limited both in bandwidth and in the amount of concurrent I/Os. As long as the number of concurrent rebuild I/Os, and the bandwidth they consume, do not exceed the predefined limits........"
"Dynamic Bandwidth Throttling: .... Note that the limits on the rebuild bandwidth and concurrent I/Os are still imposed."
On the other hand, The sentences below sound like predefined limits are imposed only when Application I/Os are occurring.
P290
"favor_application_io—Limit the number and bandwidth of rebuild IOs when application IOs are in-progress"
"dynamic_bandwidth_throttling—Limit the number and bandwidth of rebuild IOs when application IOs are in-progress or have been in a defined quiet period"
Based on my assumption, I listed them below.
No Limit:
Regardless of Application I/Os, no rebuild concurrent I/Os and bandwidth limit are applied.
Limit Concurrent I/O:
Regardless of Application I/Os, rebuild concurrent I/Os are limited to a predefined number,
and no bandwidth limit is applied.
Favor Application I/O:
Regardless of Application I/Os, rebuild I/Os wait when either I/O or bandwidth reaches their predefined thresholds,
until both I/O and bandwidth are below the their thresholds
Dynamic Bandwidth Throttling:
No rebuild concurrent I/Os and bandwidth limits are applied in case of no application I/Os occurrence,
or when Application I/Os are below both predefined thresholds of a concurrent number of I/O and bandwidth in a predefined interval.
Otherwise, rebuild concurrent I/Os are limited to a predefined concurrent number and bandwidth limit.
Many thanks,
Taizo
hakuden
16 Posts
0
December 5th, 2016 00:00
Hi
We benched.
SDS(3.3TB) x 3
Used size per SDS: 800GB
For app I/Os
100GB Volume mapped to a SDC with a mount point /mnt/scaleio command:dd if=/dev/zero of=/mnt/scaleio/file bs=1M count=3000
For failure
ifdown SDS-SDS network
We measured the size of rebuilt data in 5 min with and without app I/O app I/O by the dd command 5 times during a 5 min session The transfer rate(MB/sec) of the dd commands are also measured
*Bandwidth limits are set at the maximal value of 1048576KB for favor and dynamic policy.
We have separate NICs for SDS-SDS and SDS-SDC networks.
*The scores are not very accurate.
+No Limit
without app I/O : 108GB/5min
with app I/O : 97GB/5min
app I/O rate : btw 113 and 122MB/sec
+Limit Concurrent I/O with concurrent I/O limit of 15
without app I/O : 105GB/5min
with app I/O : 94GB/5min
app I/O rate : btw 115 and 133MB/sec
+Limit Concurrent I/O with concurrent I/O limit of 1
without app I/O : 65GB/5min
with app I/O : 60GB/5min
app I/O rate : btw 181 and 191MB/sec
+Favor Application I/O with concurrent I/O limit of 15
without app I/O : 105GB/5min
with app I/O : 95GB/5min
app I/O rate : btw 117 and 130MB/sec
+Favor Application I/O with concurrent I/O limit of 1
without app I/O : 64GB/5min
with app I/O : 61GB/5min
app I/O rate : btw 176 and 190MB/sec
+Dynamic Bandwidth Throttling with concurrent I/O limit of 15
without app I/O : 108GB/5min
with app I/O : 93GB/5min
app I/O rate : btw 123 and 130MB/sec
+Dynamic Bandwidth Throttling with concurrent I/O limit of 1
without app I/O : 65GB/5min
with app I/O : 60GB/5min
app I/O rate : btw 179 and 193MB/sec
So the results are clear.
The predefined limits are always imposed regardless of app I/Os.
The greater I/O concurrent number is the faster for rebuilding to finish.
The fewer I/O concurrent number is the better app I/O rate.
We could not measure the effect of the Dynamic Bandwidth Throttling this time with the dd command.
The quiet period, I/O and bandwidth thresholds were left default.
So if there were small app I/Os, Could the Dynamic Bandwidth Throttling have performed better?
It sounds like that we should select Limit Concurrent I/O, as far as bandwidth does not matter and set the concurrent number depending on our priority.
Or we should select Dynamic Bandwidth Throttling if the quiet period function works.
Many thanks
Taizo