Start a Conversation

This post is more than 5 years old

Solved!

Go to Solution

44798

February 26th, 2014 11:00

Md3000i will not start

I have inherited an MD3000i array that is populated with discs 7x750 SAS, one ISCSI controller and two power supplies. I cannot reach the array using storage manager. Checking the network connections I am getting no indication on the management port or the ISCSI controller ports. Connecting serial to the console I cannot enter any commands and the screen scrolls the following: àààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààà -==- Attaching interface lo0... done Adding 9768 symbols for standalone. ààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààà -==- Attaching interface lo0... done Adding 9768 symbols for standalone. ààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààà -==- Attaching interface lo0... done The system is outside of warranty some 60 days now. I have no data in the array that I care about, Please help me get this thing up and running again. Kind regards, Dan

Moderator

 • 

7.6K Posts

February 26th, 2014 13:00

Hello Dan,

lets try to remove all cables from the MD3000i and leave just the serial cable attached & see if that has any change.  Also what you can try is to pull out the controller on the MD3000i and wait about 2 minutes then insert it with the system on to see if the serial connection shows anything new.  Also what are the colors of the LEDs on the front of the MD3000i?

Please let us know if you have any other questions

Moderator

 • 

7.6K Posts

March 5th, 2014 13:00

Hello Dan,

I am not sure of how to test the memory from the controller as I know that they are regular dimms.  If you happen to have a server or desktop that the dimms can fit into you can use dell 32 bit diagnostics to test the memory.  

As for you getting a replacement MD3000i controller I know Dell is running low on selling them as they are EOL & need to keep a supply for in warranty customers.  Now with that being said if you do a search on the web for the part & see if you can find it cheaper then what we are offering.  If you do a search for this dell part# XX659 you should be able to find a replacement controller from a Dell reseller.  

Please let us know if you have any other questions.

Moderator

 • 

7.6K Posts

February 26th, 2014 11:00

Hello Dan,

When you connected the serial cable to the MD & tried to see what the controller was displaying & received the info that you did had you tried to power off the MD & then wait about 2 minutes then power it back on to see if the message has changed?

Also when you try to login to MDSM and try to connect to the MD do you get an error messages or does it just state that it can’t find the MD to manage the array.  

Please let us know if you have any other questions.

8 Posts

February 26th, 2014 11:00

Thank you for the quick response!

AS you suggested, I powered down the array and let ti sit for five minutes. Powering up the system and watching the console output gave me the same results as posted above. I am unable to find the array with MDSM. Searching for a new array of course results in nothing found.

Kind regards,

Dan

8 Posts

February 27th, 2014 01:00

The top LED on the array appears orange intermittently, cycling on/off about every 20 seconds. The middle  LED remains constantly lit and bright green in color.

The bottom and last LED is unlit.

1=Slowly flashing orange/off

2=Constant green

3=Unlit/No color

As suggested, I performed the following;

1. Disconnected all power from the array (both power supplies) completely.

2. Disconnected all network (Ethernet/ISCSI connections.

3. Removed the RAID controller completely.

After six hours;

1. Reinsert the controller

2. Reconnect both power supplies.

3. Check the console/serial connection.

4. Power on the system and watch for activity on the terminal software.

And observed the same lighting conditions as described now (in this post) and the following output to the terminal;

àà

-=<###>=-

Attaching interface lo0... done

Adding 9768 symbols for standalone.

ààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààà

-=<###>=-

Attaching interface lo0... done

Adding 9768 symbols for standalone.

àààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààààà

-=<###>=-

Attaching interface lo0... done

Repeat.

I'm sorry if this seems verbose. I do know from experience; the devil can be in the detail.

Thank you, VERY much, for your time and attention!

Kind regards,

Dan.

8 Posts

March 4th, 2014 15:00

ok, after much mucking about.. The array is responding (somewhat)  I was able to access the boot menu and was able to retrieve the following:

                          BOOT OPERATIONS MENU

1) Perform Isolation Diagnostics    10) Serial Interface Mode Menu

2) Download Permanent File          11) Display Hardware Configuration

3) Reserved                         12) Change Hardware Configuration Menu

4) Dump NVSRAM Group                13) Development Options Menu

5) Patch NVSRAM Group               14) Display Memory Error Log

6) Set Real Time Clock              15) Manufacturing Setup Menu

7) Display Board Configuration       R) Restart Controller

8) Special Services Menu             Q) Quit Menu

9) Display Exception Message

   Enter Selection: 14

Total number of multi-bit ECC errors:   1045

Total number of single-bit ECC errors:  14894

================= EXCEPTION LOG =================

Serial number:     97R000Q

Entry count:       15939

Wrap-arounds:      255

First entry time:

Current date/time: MAR-04-2014 11:47:31 PM

---- Log Entry #15909 MAR-03-2014 01:23:32 PM ----

Single-Bit ECC error at 0x1EDCB6E8, ECC Log reg 0x00101031

---- Log Entry #15910 MAR-03-2014 01:23:32 PM ----

Single-Bit ECC error at 0x1EDCB768, ECC Log reg 0x00101058

---- Log Entry #15911 MAR-03-2014 01:23:32 PM ----

Single-Bit ECC error at 0x1EDCBAE0, ECC Log reg 0x00101051

---- Log Entry #15912 MAR-03-2014 01:23:32 PM ----

Single-Bit ECC error at 0x1EDCBAF0, ECC Log reg 0x00101004

---- Log Entry #15913 MAR-03-2014 01:23:32 PM ----

Single-Bit ECC error at 0x1EDCBD40, ECC Log reg 0x00101015

---- Log Entry #15914 MAR-03-2014 01:23:32 PM ----

Multi-Bit ECC error at 0x1EDCBD48, ECC Log reg 0x00101160

---- Log Entry #15915 MAR-03-2014 01:23:55 PM ----

Single-Bit ECC error at 0x1EDCBED8, ECC Log reg 0x00101031

---- Log Entry #15916 MAR-03-2014 01:23:55 PM ----

Single-Bit ECC error at 0x1EDCBFA8, ECC Log reg 0x00101031

---- Log Entry #15917 MAR-03-2014 01:23:55 PM ----

Single-Bit ECC error at 0x1EDCC008, ECC Log reg 0x00101034

---- Log Entry #15918 MAR-03-2014 01:23:55 PM ----

Single-Bit ECC error at 0x1EDCC2C0, ECC Log reg 0x00101025

---- Log Entry #15919 MAR-03-2014 01:23:55 PM ----

Single-Bit ECC error at 0x1EDCC588, ECC Log reg 0x00101051

---- Log Entry #15920 MAR-03-2014 01:23:55 PM ----

Single-Bit ECC error at 0x1EDCCA80, ECC Log reg 0x00101052

---- Log Entry #15921 MAR-03-2014 01:23:55 PM ----

Multi-Bit ECC error at 0x1EDCCA88, ECC Log reg 0x00101166

---- Log Entry #15922 MAR-03-2014 01:24:19 PM ----

Single-Bit ECC error at 0x1EDCCAE0, ECC Log reg 0x00101051

---- Log Entry #15923 MAR-03-2014 01:24:19 PM ----

Single-Bit ECC error at 0x1EDCCAF0, ECC Log reg 0x00101004

---- Log Entry #15924 MAR-03-2014 01:24:19 PM ----

Single-Bit ECC error at 0x1EDCCD40, ECC Log reg 0x00101015

---- Log Entry #15925 MAR-03-2014 01:24:19 PM ----

Multi-Bit ECC error at 0x1EDCCD48, ECC Log reg 0x00101160

---- Log Entry #15926 MAR-03-2014 01:24:42 PM ----

Single-Bit ECC error at 0x1EDCCE80, ECC Log reg 0x00101049

---- Log Entry #15927 MAR-03-2014 01:24:42 PM ----

Single-Bit ECC error at 0x1EDCD2C0, ECC Log reg 0x00101025

---- Log Entry #15928 MAR-03-2014 01:24:42 PM ----

Single-Bit ECC error at 0x1EDCD588, ECC Log reg 0x00101051

---- Log Entry #15929 MAR-03-2014 01:24:42 PM ----

Single-Bit ECC error at 0x1EDCDA80, ECC Log reg 0x00101052

---- Log Entry #15930 MAR-03-2014 01:24:42 PM ----

Multi-Bit ECC error at 0x1EDCDA88, ECC Log reg 0x00101166

---- Log Entry #15931 MAR-03-2014 01:25:05 PM ----

Single-Bit ECC error at 0x1EDCDAE0, ECC Log reg 0x00101051

---- Log Entry #15932 MAR-03-2014 01:25:05 PM ----

Single-Bit ECC error at 0x1EDCDAF0, ECC Log reg 0x00101004

---- Log Entry #15933 MAR-03-2014 01:25:05 PM ----

Single-Bit ECC error at 0x1EDCDD40, ECC Log reg 0x00101015

---- Log Entry #15934 MAR-03-2014 01:25:05 PM ----

Multi-Bit ECC error at 0x1EDCDD48, ECC Log reg 0x00101160

---- Log Entry #15935 MAR-03-2014 01:25:28 PM ----

Multi-Bit ECC error at 0x1EDCE008, ECC Log reg 0x00101121

---- Log Entry #15936 MAR-03-2014 01:25:51 PM ----

Single-Bit ECC error at 0x1EDCE588, ECC Log reg 0x00101051

---- Log Entry #15937 MAR-03-2014 01:25:51 PM ----

Single-Bit ECC error at 0x1EDCEA80, ECC Log reg 0x00101052

---- Log Entry #15938 MAR-03-2014 01:25:51 PM ----

Multi-Bit ECC error at 0x1EDCEA88, ECC Log reg 0x00101166

Moderator

 • 

7.6K Posts

March 5th, 2014 06:00

Hello Dan,

With looking at the output that you posted it looks like there is a memory issue with the controller. Normally what we would do if the system is under warranty is to remove the raid controllers from the MD3000i & then remove the raid battery from both controllers as well and let the controllers sit without their battery for about 5 minutes. After that put the battery back into the controllers and then insert the controllers back into the MD3000i & boot it up.

Now if that does not resolve the memory issues you can reseat the memory on the controllers. But if you choose to reseat the memory on the controllers there is a chance that the controller may not boot up again so, so we don’t recommend do this unless all options have been tested. Now if you do choose to reset the memory & still getting the memory error then the controller will need to be replaced as it will fail again when it hits the limit of the memory that is causing the error.

Please let us know if you have any other questions.

8 Posts

March 5th, 2014 08:00

Thanks Sam,

Is there any way to isolate/localize the error to determine is the faulting memory is located in the DIMM or the CPU?

I performed the tests as you described and have to accept the controller or RAM needs replacement.

My warranty expired on February 2nd, is there any chance Dell might take pity on my luck; possibly assist me in getting a replacement at a discounted rate? Whom shall I contact?

No Events found!

Top