Unsolved
This post is more than 5 years old
62 Posts
0
1039
November 28th, 2011 14:00
BCV Devices are Taking too much time in Splitting in Background
Gents,
One of our SYMM(DMX-4) is taking too much time to do the splitting in Background.Background splitting is running for the last 5 days.Still some of the devices are in near to 95~98%.PSE Lab dialed in saying that one of the Cache Partition write pending Tracks is more than the 50% of that specific cache Partition write Pending Threshold.And thats the reason all the clone Operation is stuck.
Even we checked the symstat for this BCV devices where it seems that very less no of write pending tracks on the BCV but destaging is almost 0 which makes the backgrouns split almost freeze.
Anybody has any idea how to resolve this issues???
root@chsapp11-/usr/symcli/bin> ./symmir -g chsapp11BCV query -percent -bg
Device Group (DG) Name: chsapp11BCV
DG's Type : RDF1
DG's Symmetrix ID : 000290103889
Init/
Standard Device BCV Device State Split
-------------------------- --------------------------------- ------------ -----
Inv. Inv.
Logical Sym Tracks Logical Sym Tracks STD <=> BCV (%)
-------------------------- --------------------------------- ------------ -----
DEV001 1E1E 0 BCV001 21B4 * 0 Split 100%
DEV002 1E20 0 BCV002 21B6 * 0 Split (bg) 98%
DEV003 1E22 0 BCV003 21B8 * 0 Split (bg) 99%
DEV004 1E24 0 BCV004 21BA * 0 Split 100%
DEV005 1C5E 0 BCV005 21BC * 0 Split (bg) 98%
DEV006 1C70 0 BCV006 21CE * 0 Split (bg) 98%
DEV007 1C82 0 BCV007 21E0 * 0 Split (bg) 98%
DEV008 1C94 0 BCV008 21F2 * 0 Split (bg) 98%
DEV009 1CA6 0 BCV009 2204 * 0 Split (bg) 98%
DEV010 1CB8 0 BCV010 2216 * 0 Split (bg) 98%
DEV011 1CCA 0 BCV011 2228 * 0 Split (bg) 98%
DEV012 1CDC 0 BCV012 223A * 0 Split (bg) 98%
DEV013 1CEE 0 BCV013 224C * 0 Split 100%
DEV014 1F9E 0 BCV014 2260 * 0 Split 100%
DEV015 1F9F 0 BCV015 2261 * 0 Split 100%
DEV017 2786 0 BCV017 2265 * 0 Split (bg) 98%
DEV016 2AA6 0 BCV016 2AB9 * 0 Split (bg) 96%
DEV018 2C73 0 BCV018 2399 * 0 Split (bg) 94%
DEV019 2BC5 0 BCV019 2397 * 0 Split (bg) 98%
DEV020 2331 0 BCV020 2344 * 0 Split 99%
Total ------ -------
Track(s) 0 0
MB(s) 0.0 0.0
Legend:
(*): The paired BCV device is associated with this group.
(bg): The paired BCV device is splitting in the background.
root@chsapp11-/usr/symcli/bin> ./symstat -type MEMIO -g chsapp11BCV -bcv -i 10 -c 15
DEVICE Tracks/sec % Dev
17:11:54 WP Tracks Prefchd Destgd WPmax
17:12:07
17:12:18 BCV013 (224C) 0 0 0 0
BCV016 (2AB9) 1 0 0 0
------ ------ ------ ------
1 0 0 0
17:12:30 BCV005 (21BC) 0 0 0 0
DEVICE Tracks/sec % Dev
17:12:42 WP Tracks Prefchd Destgd WPmax
17:12:42 BCV016 (2AB9) 1 0 0 0<-------------------------No Destaging
17:12:54 BCV014 (2260) 0 0 0 0
17:13:06 BCV011 (2228) 0 0 0 0
17:13:18
17:13:30 BCV007 (21E0) 1 0 0 0<-------------------------No Destaging
BCV008 (21F2) 1 0 0 0<-------------------------No Destaging
BCV016 (2AB9) 4 0 0 0<-------------------------No Destaging
------ ------ ------ ------
6 0 0 0
DEVICE Tracks/sec % Dev
17:13:42 WP Tracks Prefchd Destgd WPmax
17:13:42 BCV016 (2AB9) 2 0 0 0<-------------------------No Destaging
17:13:53 BCV002 (21B6) 0 0 0 0
BCV015 (2261) 0 0 0 0
BCV016 (2AB9) 4 0 0 0<-------------------------No Destaging
------ ------ ------ ------
4 0 0 0
17:14:05
17:14:17
17:14:29 BCV016 (2AB9) 1 0 0 0<-------------------------No Destaging
BCV018 (2399) 1 0 0 0<-------------------------No Destaging
------ ------ ------ ------
2 0 0 0
DEVICE Tracks/sec % Dev
17:14:41 WP Tracks Prefchd Destgd WPmax
17:14:41
17:14:53 BCV003 (21B8) 0 0 0 0<-------------------------No Destaging
BCV018 (2399) 1 0 0 0<-------------------------No Destaging
------ ------ ------ ------
1 0 0 0
Quincy561
1.3K Posts
0
December 1st, 2011 12:00
Why are you using cache partitions? Also are these real BCVs (not clones?)
If they are real true BCVs, I can't think of a reason why you would use cache partitions unless they have nothing to do with the BCV activity.
Scott_Dugan
26 Posts
0
December 1st, 2011 16:00
Are the BCV Devices raid-5?
It looks like the background split is slow due to cache utilization.
You might need to pend some activites and see if it speeds up. maybe suspend the link and see if that helps.
Any chance you can get a few dollars to purchase some cache boards from management, last resort.
MorbidAngel1
62 Posts
0
December 1st, 2011 16:00
HI Quincy,
We are using Cache Partition just to get some dedicated Resource for the Mainframe Devices.The Devices are True BCV's.
The BCV Devices are in the Default partition where most of the Devices resides.
Hi Scott,
Yes.These BCV Devices are all RAID-5 Devices.The Problem is only a certain cache Partition is going more than 50% whereas the Other Cache Partition is still healthy and the Overall Array Cache is Healthy.
root@chslyp10-/usr/symcli/bin> ./symqos -sid 3889 list -cp -settings -usage
Symmetrix ID: 000290103889
Cache Partitioning : Enabled
Number of Partitions : 4
Max Num of Partitions : 8
Min Allowed Target % : 10
Max Allowed Target % : 90
XRC Partition State : Disabled
XRC Partition Name : N/A
Empty Partition State : Preserve
Time of Last Modification : 01:49:08 PM on Fri Nov 25,2011
Cache Slots
Min Tgt Max WP Time Device ------------------ Used
Partition Name (%) (%) (%) (%) (s) Count WP Used (%)
----------------- --- --- --- --- ----- ------ -------- -------- ----
DEFAULT_PARTITION 0 47 100 80 300 8103 622947 2135370 71
ACP 0 10 10 80 300 31 20 53 0
mainframe 23 23 23 80 300 2780 13433 690807 23
BIGWASR 5 20 20 80 300 540 51249 200371 7
Quincy561
1.3K Posts
0
December 1st, 2011 18:00
Also I would consider changing the minimum % of the mainframe partition to a much lower value, even zero, and increasing the time value instead. You could make it several thousand seconds. This way if the MF partition isn't using the cache at all, it can give it up, but it will keep it for a long time if it is using it.
It would be interesting to look at the DCP partition metrics.
Quincy561
1.3K Posts
0
December 1st, 2011 18:00
RAID5 BCVs are NOT true BCVs, they are clones. So your split is not a split, but a clone activate. A clone activate is different than a BCV split of a mirror.
A clone activate should be very quick.
I would suggest you open a case for this.
Also I highly recommend switching to native clone commands from symmir commands so there is no translation.