Equallogic PS6100 not Start after power failure

Question

I have problems starting Equallogic in my production environment.

Please somebody help

Thanks!!!!!!!

#########################################################
# #
# Dell (tm), Inc. Storage Array #
# Copyright 2001-2011 #
# #
# Controller Information: #
# Part=70-0477 Rev=A00 SN= 13740028U000S ECO=C00 #
# #
#########################################################
Bootloader Version 3.9.1 (SWINT Rev:1)
Compiled on Wed Nov 16 14:22:40 EST 2011
(type h for help)
Enter Ctrl-P for boot prompt

Executing bootcmd0 [dload sd primary/eqlstor.gz]
0X80231000/11912 0X85f08100/40013432 entrypt 80231000

Executing bootcmd1 [run]
cpu_online_map=ffff, userapp_cpu_map ffff
psb_os_active_mask=0, psb_os_mask=0
boot1_info: userapp_cpu_map=ffff, psb_os_cpu_map=0
cpu_online_map = 0xffff
Jumping to the application... 0x80231000
------------------------------------------------------------
Preparing ffff bitmask of cpus to run
No network device to cleanup
count = 16, total = 16
All slave cpus (16) ack'ed userapp init
count = 4, total = 4
All slave cpus (4) ack'ed message ring init

Dell Inc. Storage Array

Copyright 2001-2015 Dell Inc.
SP:1.01:mips_pss_init.c:363:INFO:28.2.107:Control module in slot 0 with serial number CN-09PDDY-13740-28U-000S is designated as active.
MFS set up
Building databases...
SP:1579818120.94:emm.c:1333:INFO:28.2.6:Enclosure serial number: CN-0WK7G2-70821-25L-05P5-A01.
Restoring password file...
SP:1579818126.27:emm.c:2363:ERROR:28.4.47:Critical health conditions exist.
Correct immediately before they affect array operation.
Critical hardware component failure.
There are 1 outstanding health conditions. Correct these conditions before they affect array operation.
SP:1579818126.27:emm.c:355:ERROR:28.4.85:Critical hardware component failure, as shown next.
C2F power module has insufficient hold time.
SP:1579818126.27:cache_driver.cc:1056:WARNING:28.3.17:Active control module cache is now in write-through mode. Array performance is degraded.
Thu Jan 23 17:22:14 EST 2020
Jan 23 17:22:14 init: kernel security level changed from 0 to 1

PS Series Storage Arrays
Unauthorized Access Prohibited

login: SP:1579818138.94:emm.c:2363:WARNING:28.3.51:Warning health conditions currently exist.
Correct these conditions before they affect array operation.
Control module was removed from array.
There are 1 outstanding health conditions. Correct these conditions before they affect array operation.
SP:1579818138.94:emm.c:1435:ERROR:28.4.50:Control module in slot 1 is not functioning or not installed.
MgmtExec:1579818142.83:IoManager.cc:452:ERROR:7.4.1:No valid storage pool found.

dwilliam62 · Answer

Hello,

If you do decide to power down, first login and issue a shutdown command.

But before that I would suggest logging in and running

GrpName>member select MEMBERNAME show controllers

GrpName>member select MEMBERNAME show

The first command will show if both controllers are online and synced . The second command will show what the current health status is and may provide more info on the error condition(s) that exist.

It should show you that the C2F (Cache-to-flash) "battery" has failed. It's not technically a battery but it's part of the controller.

So that controller will need to be replaced.

Based on the boot up sequence the firmware is old as well.

Regards,

Don

DELL-Sam L · Answer

Hello Rvilla123,

Is your system still under warranty? What I would recommend is shutting the unit down and unplugging all power and leave it down a full 5 minutes. Cheack & reseat all cables while the system is off. Once you power the system back on see if the controller is still showing as missing or present & ready.

Please let us know if you have any other questions.

RVilla123 · Answer

Hello guys, after 2 days this is the state of SAN :(, any idea?

Thanks.

4928:30:netmgtd:27-Jan-2020 08:12:34.570030:rcc_task.c:450:INFO::25.2.6:CLI: Account grpadmin logged out.
ATTENTION!
A critical health condition exists on the array. User intervention is required.
Please call your support provider.
Once the problem is repaired the array MUST be rebooted.

PS Series Storage Arrays
Unauthorized Access Prohibited

login: grpadmin
Password:
Last login: Mon Jan 27 07:12:25 2020 on console

Welcome to Group Manager

LV log has been damaged. Contact the support provider for your PS Series array.

The storage array is still initializing. Limited commands will be available unt
il the initialization is complete. Please try again later.

CLI> 5140:36:netmgtd:27-Jan-2020 12:20:29.980036:rcc_util.c:1024:INFO::25.2.9:CLI: Login to account grpadmin succeeded, using local authentication. User privilege is group-admin.

CLI> member select Arrayconix
The storage array is still initializing. Limited commands will be available unt
il the initialization is complete. Please try again later.

CLI>

dwilliam62 · Answer

Hello,

At this point the only resolution will involve calling support and opening a case. If you don't have a service contract you might be able to open a one-time case for a fee.

The LV log is an internal database the array uses to track blocks. The array will never complete the boot up process until that is resolved.

Do NOT power off this array in an attempt to fix it.

Regards,

Don

dwilliam62 · Answer

Hello,

re: Power off. That doesn't help unfortunately and can make things worse.

re: Call. If it is only a problem in the LV the odds are very good that it can be fixed. If there is also an underlying hardware problem that will have to be addressed as well.

The real question is how much is that data worth? Do you have a backup? A reset would create a new blank LV log then you could do a full restore.

Regards,

Don

RVilla123 · Answer

Thanks Don, for your reply.

We have turned off this array several times.

this is the actual state after the last shutdown.

You sincerely believe that this problem can be solved in a support call?

Or do you think it is a waste of time and money?.

Thank you again.

RVilla123 · Answer

Hello, Don. Thanks again.

We only need one user's virtual machine. This virtual machine only has the history of a file with extremely important Excel macros that has been developed for many years.

The rest is the local backup of another (EVA´s) SAN, which is replicated in the cloud and it is not important to recover it.

But the damn virtual machine has no local or cloud backup.

dwilliam62 · Answer

Hello,

You are very welcome. I am very happy to help. I would imagine it would cost a lot to recreate all those scripts/macros.

I would still suggest you try to get it looked at. They'll need to have a lftp server on the iSCSI subnet to offload the LV log. But they will help you with that.

Regards,

Don

dr.kiev · Answer

It is possible to recover nessesary data from disks outside server, connected via any HBA contorller and remote access. After that you can do whatever you want with equallogic.

dwilliam62 · Answer

Hello Dr.Kiev,

You keep posting these cryptic replies to any EQL problem but always without details. The LV log is a database that exists on the RAIDset. So if you just access the disks without the EQL firmware constituting the RAIDset you can't gain access to the volume data with any server or HBA.

Just accessing the blocks on the drives won't get you access to the FILES, in this case the VM. The data is not stored sequentially. The LV log says what LBAs for each volume are stored on what 15MB pages.

You appear to be representing a recovery company. Is that correct?

Regards,

Don

dr.kiev · Answer

Yes, you are right , i am raid recovery expert from data recovery company , specializing on dell products. I do not hide this info.

And yes, Equallogic is tricky to solve out of the box, but possible. That i am doing every day.

And one more yes, to gain access to data need to solve couple of issue including decoding "block level" database which locates exactly on every drive at the beginning (Meta data area) . Difficult but solvable.

What kind of details do you want to know more?

dwilliam62 · Answer

Hello,   Thank you.  In this case it's not just about reading that information, it contains errors. Which much be fixed.   Are you saying you can repair that database as well?  Don

dwilliam62 · Answer

Hello,

Sorry to hear that. Where are you located? it's mostly in the Americas that the OOW support process is supported at this time. It would have to be done via technical support, not "commercial"

Regards,

Don

RVilla123 · Answer

Thank you very much to all.

If someone can give me any additional instruction I will be very grateful.

I tell you the progress ...

We contacted DELL, but there was no satisfactory answer.

First in the support area they indicate that we must communicate with the commercial area and then in the commercial area they indicate that we must communicate with the support area and we stay in a loop that they could not solve.

We are looking for support with a third party.

RVilla123 · Answer

We are in Colombia

EqualLogic

Equallogic PS6100 not Start after power failure

Was this post helpful?