Replacing a faulty Raid controller Powervault MD3620i

Question

Hi.

We have an issue with RAID Controller 0 on our MD3620i.

It has status failed with service action (removal) allowed: yes.

Question: What is the best prosedure for replacing this RAID controller with a refurbished one?

Does the replacement RAID controller need to be wiped clean of any settings? In the Recovery Guru procedure it is mentioned that we can release the faulty Raid controller, take it out and wait 1 minute. Then we put the refurbished one back in and configure this as online. What about the existing settings and the storage disks? What is the best prosedure to keep all settings?

We also have an MD3600i Storage Array with similar controller. Is it possible to wipe one of the controllers in this and instead use it in our MD3620i Storage Array?

Best regards

Vidar

DELL-Charles R · Accepted Answer

Hello vide123,

Best POA (plan of action)

Purchase new storage
Connect new storage to shared host
Move data through the file system
1. Something like vMotion

POA to update firmware to attempt to correct common issues

If possible create a full backup in case of lost access to data
These steps are to try to narrow down the cause of the issue
Note that it is possible to lose access to storage until corrective work is done with paid support from Support

Preparatory work

Download latest version of MDSM to local storage
1. https://www.dell.com/support/home/en-us/drivers/driversdetails?driverid=r9g1x&oscode=w12r2&productcode=powervault-md3620i
2. Extract ISO to local storage
3. 1. Do not attach to a VM as that does not work
Download and extract Controller and HDD updates to local MDSM server
1. https://www.dell.com/support/home/en-us/drivers/driversdetails?driverid=2dxm8&oscode=w12r2&productcode=powervault-md3620i
2. https://www.dell.com/support/home/en-us/drivers/driversdetails?driverid=4cp5x&oscode=w12r2&productcode=powervault-md3620i
Note down IP addresses used for Out Of Band (OOB) connections
1. RC1 is 172.21.100.39
Uninstall MDSM from management host
Reboot management system (VM/Host)
Install new MDSM from local host storage
Readd storage with both OOB addresses noted down above
1. RC0 address will fail
Move the bus mode switch on the MD3620i to the up position
1. This is on the front of the system in the lower left hand side under the status lights
2. Verify it is up and stays up until the end of time
3. It comes from the factory in the down position
Prepare for a maintenance window
1. If all goes well,
2. 1. Around 2-4 hours

When you have the maintenance window,

Put ESX hosts into maintenance mode or turn them off or stop all VMs/operations on the storage
Offline the storage once the above is complete by turning off the power supplies
Monitor until lights on back of controllers go off
Unseat RC0 (Top)
Place it to the side as we will not use it for a while
1. Note the part number on the controller
Move RC1 (bottom) to the top slot with attached cables
Make sure to seat fully/correctly
Turn on storage system
Verify data is seen and system is seen in MDSM
1. Note that the IP address may change over to RC0 address automatically
If MDSM is unable to access the storage in about 15 minutes,
1. Reverse the controller swap , put it back in original slot, and verify data access
IF the storage comes back online and you can see the config,
1. Open MDSM to the first window with Devices/Setup tabs
2. Right click on the name of the array
3. Click "Execute Script…"
4. In the top text box,
5. 1. Type "set storageArray redundancyMode=simplex;"
6. In the file bar above,
7. 1. Click Tools -> Verify and Execute
8. If the command completes successfully,
9. 1. Close out of the box, do not save
  2. Open second window of MDSM that starts with Summary
  3. Click on the Support tab
  4. Click on Download Firmware
  5. Select "Download RAID controller module firmware"
  6. Click OK
  7. If a window pops up about "Event log issues",
  8. 1. Cancel out of the box
    2. Run a new set of logs
    3. Open "View Event Log"
    4. Click Clear All… to clear the logs
    5. Then repeat the Download Firmware steps above
  9. Use the button Select File…
  10. Navigate the folders for the FW for a MD3620i
  11. Make sure "files of type" shows Compatible Firmware Files
  12. Select the only file in the folder which should say 08.20.24.60
  13. Click ok on the selection
  14. Click on the check box to Transfer NVSRAM
  15. Use the new Select File… button
  16. Navigate the folders for the FW for a MD3620i
  17. Make sure "files of type" shows Compatible Firmware Files
  18. Select the only file in the folder which should have a different file name
  19. Click Transfer…
  20. Whole process should take less than 30 minutes
  21. Once it says it is complete,
  22. 1. Go back to Download Firmware
    2. Choose Download physical disk firmware
    3. Use menus to find and chose matching FW for "AS0D" and "EF06"
    4. Follow steps to select all drives and update
  23. Once complete with those updates,
  24. 1. Open MDSM to the first window with Devices/Setup tabs
    2. Right click on the name of the array
    3. Click "Execute Script…"
    4. In the top text box,
    5. 1. Type "set storageArray redundancyMode=duplex;"
    6. In the file bar above,
    7. 1. Click Tools -> Verify and Execute
    8. When complete the storage should go into a yellow state with !
    9. While the system is running,
    10. 1. Insert the replacement controller into the bottom slot
    11. Monitor for up to 10-15 minutes
    12. 1. Controller may reboot 2-3 times to match FW, cache, and settings
    13. Run new logs if issues are not gone

vide123 · Answer

We have replaced the faulty RAID controller with a refurbished one wiping all data from the refurbished one before the change of controllers.

But when we boot the replacement controller has exact same fault as the one we replaced.

On the back of the controller the iSCSI ports shows activity with orange and green leds lit. The network interface port shows no activity (no lights)

Status LEDS on the Controller is Grenn for power. Orange for warning. Blue for service mode, green for circle with arrow and finally orange for battery. All of theese leds are lit.

What can cause this problem? Any suggestions?

DELL-Charles R · Answer

Hello vide123,

I'm sorry to see the refurb controller did not resolve this issue for you. Let's get a look at the logs.

How to gather Support Information - MD32xx + MD34xx + MD36xx + MD38xx (SAS + I + f)

https://www.dell.com/support/kbdoc/en-us/000127694/how-to-gather-the-support-logs-of-powervault-md32xx-md34xx-md36xx-and-md38xx

When you have the log please upload here under the service tag of the storage unit.

Then please Private Message me the Service tag, as well as your company name.

https://upload.dell.com/

DELL-Charles R · Answer

Hello vide123,   I received your private message with the service tag and I will collect the logs to review. I will update you when I have more information.

DELL-Charles R · Answer

Hello vide123,

I wanted to provide you an update.

The running controller 1 (the bottom controller) is on very old firmware 07.80.41.60 (12 years old), we are up to 08.20.24.60, and it keeps kicking the replacement controller 0 offline.

It may be best to contact support and set up a service case with a Pay as you need (PAYN) contract if it is available to you so that an Engineer can do a remote session and work on this with you.

If you can get a 2GB controller with old firmware (07.xx.xx.xx) you may be able to get the controller to stay on if they are closer in firmware.

I'll have some steps you can try as best effort to send out probably tomorrow.

If possible you may try to backup and critical data.

vide123 · Answer

Hi, and thanks for the steps for how to best fix this issue.

We will have a look into theese steps and try to run through them.

Maybe use payed support as suggested.

Thanks for your help.

Best regards

Vidar

PowerVault

Replacing a faulty Raid controller Powervault MD3620i

Was this post helpful?