Unsolved
This post is more than 5 years old
68 Posts
0
952
November 10th, 2016 18:00
Kernel message after ScaleIO upgrade 2.0.0.2 -> 2.0.1.0 (maybe SMART related?)
Hello,
we have two ScaleIO infrastructures: one licensed for production and one with evaluation license used only for testing.
The licensed one was installed on physical hardware with ScaleIO version 2.0.1.0 and I didn't notice any strange behaviour.
The Evaluation license is installed on three HyperV VMs. The evaluation infrastructure was running ScaleIO version 2.0.0.2 and today I decided to switch to version 2.0.1.0. After the upgrade in the kern.log I can see these logs:
Nov 11 02:35:05 sio03 kernel: [ 1072.557719] sd 3:0:0:0: [storvsc] Sense Key : Illegal Request [current]
Nov 11 02:35:05 sio03 kernel: [ 1072.557740] sd 3:0:0:0: [storvsc] Add. Sense: Invalid command operation code
Nov 11 02:35:05 sio03 kernel: [ 1072.557823] sd 3:0:0:0: [storvsc] Sense Key : Illegal Request [current]
Nov 11 02:35:05 sio03 kernel: [ 1072.557827] sd 3:0:0:0: [storvsc] Add. Sense: Invalid command operation code
Nov 11 02:35:05 sio03 kernel: [ 1072.558264] sd 3:0:0:0: [storvsc] Sense Key : Illegal Request [current]
Nov 11 02:35:05 sio03 kernel: [ 1072.558268] sd 3:0:0:0: [storvsc] Add. Sense: Invalid command operation code
Nov 11 02:35:05 sio03 kernel: [ 1072.558662] sd 3:0:0:0: [storvsc] Sense Key : Illegal Request [current]
Nov 11 02:35:05 sio03 kernel: [ 1072.558665] sd 3:0:0:0: [storvsc] Add. Sense: Invalid command operation code
Nov 11 02:36:05 sio03 kernel: [ 1132.564701] sd 3:0:0:0: [storvsc] Sense Key : Illegal Request [current]
Nov 11 02:36:05 sio03 kernel: [ 1132.564731] sd 3:0:0:0: [storvsc] Add. Sense: Invalid command operation code
Nov 11 02:36:05 sio03 kernel: [ 1132.565070] sd 3:0:0:0: [storvsc] Sense Key : Illegal Request [current]
Nov 11 02:36:05 sio03 kernel: [ 1132.565075] sd 3:0:0:0: [storvsc] Add. Sense: Invalid command operation code
Nov 11 02:36:05 sio03 kernel: [ 1132.565647] sd 3:0:0:0: [storvsc] Sense Key : Illegal Request [current]
Nov 11 02:36:05 sio03 kernel: [ 1132.565655] sd 3:0:0:0: [storvsc] Add. Sense: Invalid command operation code
Nov 11 02:36:05 sio03 kernel: [ 1132.566142] sd 3:0:0:0: [storvsc] Sense Key : Illegal Request [current]
Nov 11 02:36:05 sio03 kernel: [ 1132.566148] sd 3:0:0:0: [storvsc] Add. Sense: Invalid command operation code
Nov 11 02:37:05 sio03 kernel: [ 1192.563729] sd 3:0:0:0: [storvsc] Sense Key : Illegal Request [current]
Nov 11 02:37:05 sio03 kernel: [ 1192.563758] sd 3:0:0:0: [storvsc] Add. Sense: Invalid command operation code
Nov 11 02:37:05 sio03 kernel: [ 1192.564098] sd 3:0:0:0: [storvsc] Sense Key : Illegal Request [current]
Nov 11 02:37:05 sio03 kernel: [ 1192.564102] sd 3:0:0:0: [storvsc] Add. Sense: Invalid command operation code
Nov 11 02:37:05 sio03 kernel: [ 1192.565119] sd 3:0:0:0: [storvsc] Sense Key : Illegal Request [current]
Nov 11 02:37:05 sio03 kernel: [ 1192.565124] sd 3:0:0:0: [storvsc] Add. Sense: Invalid command operation code
Nov 11 02:37:05 sio03 kernel: [ 1192.565342] sd 3:0:0:0: [storvsc] Sense Key : Illegal Request [current]
Nov 11 02:37:05 sio03 kernel: [ 1192.565347] sd 3:0:0:0: [storvsc] Add. Sense: Invalid command operation code
As you can see I get 8 new messages exactly every minute. I think that the latest version of SIO SDS sends some SCSI command to the disk that aren't supported by the HyperV hv_storvsc driver. If I stop the sds-2.0.10000.2072 process on a node the annoying messages in the kern.log of the specific node stop immediately. This happens also with all frontend volumes unmapped.
Anyway the SIO enviroment works perfectly, I noticed only this annoying behaviour.
Someone can explain what SCSI command SIO sends exactly every minute that seems unsupported from HyperV hv_storvsc driver? Reading the release note of the ScaleIO version 2.0.1.0 I noticed that S.M.A.R.T. disk health check has been added. I tried to query the disk for smart data using smartctl and I get a similar message in the kern.log, someone knows if S.M.A.R.T. commands are sent exactly every minute from ScaleIO? In this case is it possible to disable S.M.A.R.T. monitoring in ScaleIO?
Thanks in Advance,
Davide
pawelw1
306 Posts
2
November 11th, 2016 09:00
Hi Davide,
I believe you are right - in ScaleIO 2.0.1 (Linux bare metail solutions) S.M.A.R.T. is enabled by default and MDM will query each SDS device every minute. I couldn't find any command that would disable it - you can try to remove 'smartmontools' package that comes with LIA and is used for the monitoring purposes, but I don't know if that will work - if it's a test environment, you can give it a try and see if it helps.
As to what commands exactly are being sent - I have no idea. I don't think that MDM sends any specific opcodes to test the drivers, rather relies on 'smartctl' capabilities, so you'd have to go through the smartmontools code to be sure... :-)
Hope that helps!
Cheers,
Pawel
c0redump
68 Posts
1
November 11th, 2016 10:00
Hi Pawel,
thanks for the hint, I think you are right, I investigated a little and I discovered that sds process sds-2.0.10000.2072 relies on the dynamic library smart.so loaded from the lia process lia-2.0.10000.2072: in fact with the lia process stopped I can't see that message anymore.
I know that normally ScaleIO is installed on physical hardware also in hyperconverged scenarios but do you think that is possible, in future releases of ScaleIO, to plan a configuration parameter to switch off the S.M.A.R.T. checking? In virtual environments and sometimes in physical ones (with some specific disk or controllers), S.M.A.R.T. commands are not handled correctly and the result is a flood of the OS logs.
Kind regards,
Davide
pawelw1
306 Posts
0
November 12th, 2016 00:00
Hi Davide,
Let me take it to the Dev team and see what they say.
Cheers,
Pawel