Start a Conversation

Unsolved

This post is more than 5 years old

3096

December 6th, 2013 07:00

NAS (Celerra) 6.0.40 gui not coming up

I am trying to decommission the storage and wipe the drives. Yesterday I was able to access the Unisphere most of the day. After a few hours the connection hung and since them I have not been able to get back into it.

Yesterday as part of the wipe I removed all shares, luns, disk groups (EXECEPT I kept the raid group for the "NAS/OS"). I then regrouped all the disks into 2 arrays so I could wipe them (again leaving the NAS/OS raid group alone). However when I went back into the Unisphere it had hung and i have been unable to get it to come up so I can create a share that to my raid groups.

I have powered off the data movers and the nas and I am still unable to get it to work.

Any suggestion on what has gone wrong?

p.s. i can still access the navisphere.

Thanks

bg

----------------------

[nasadmin@cotys ~]$ su -

Password:

[root@cotys ~]#

[root@cotys ~]# /nasmcd/sbin/getreason;  /nas/tools/nas_summary; /nas/bin/nas_checkup;

10 - slot_0 primary control station

0 - slot_2 reset

0 - slot_3 reset

nas_summary v2.1

cat: /nas/server/servers: No such file or directory

cat: /nas/server/servers: No such file or directory

cat: /nas/server/servers: No such file or directory

cat: /nas/server/servers: No such file or directory

cat: /nas/server/vdm/vdms: No such file or directory

cat: /nas/server/vdm/vdms: No such file or directory

cat: /nas/server/vdm/vdms: No such file or directory

cat: /nas/server/vdm/vdms: No such file or directory

cat: /nas/volume/symms: No such file or directory

cat: /nas/volume/filesys: No such file or directory

grep: /nas/server/slot_*/netd: No such file or directory

grep: /nas/server/slot_*/export: No such file or directory

grep: /nas/site/cshosts: No such file or directory

/nas/tools/nas_summary: line 349: [: -gt: unary operator expected

Collecting Replication info...Please Wait

#####################################################

System type: NS20FC      (Gateway)

Version: 6.0.40-8

#####################################################

Number of data movers: 0         VDMs: 0

#####################################################

Number of arrays: 0

NAS: NAS NAS

#####################################################

Filesystems:            UXFS:1  Checkpoint:1            Total:0

Number of disk vols (LUNs):      1

#####################################################

Replication mode: RepV2         Replication license not enabled

# of Interconnects: 1           Remote Celerras: 1

#####################################################

Features in use:        CIFS: False     iSCSI: *True*   NFS: False

                        Rep: False      SRDF: False     DHSM: *True*

#####################################################

DBCHK State: Errors found!

Check Version:  6.0.40.8

Check Command:  /nas/bin/nas_checkup

Check Log    :  /nas/log/checkup-run.131206-141509.log

-------------------------------------Checks-------------------------------------

Control Station: Checking if NBS clients are started....................... Pass

Control Station: Checking if NBS configuration exists...................... Pass

Control Station: Checking if NBS devices are accessible.................... Fail

Control Station: Checking if NBS service is started........................ Fail

Control Station: Checking if NAS partitions are mounted.................... Pass

Blades         : Checking status........................................... Warn

--------------------------------------------------------------------------------

One or more errors have occurred. Follow the instructions provided to correct

the problem and try again.

------------------------------------Warnings------------------------------------

Blades : Check status

Warning HC_DM_18800115755: The following Blades are rebooting:

         * server_2

         * server_3

         Because of this, several other Blade checks may return a

         question mark "?".

Action : Please wait a few minutes to allow the Blade to finish

         rebooting. Then use the /nasmcd/sbin/getreason command to obtain the

         reason code and ensure that it completes the boot cycle (reason code

         should be 4 or 5). Once the boot cycle is complete, re-run nas_checkup

         so that all Blade checks can be completed. If the Blade does not

         complete the boot cycle, use server_log to check for errors on the

         Blade. Escalate this issue through your support organization if

         necessary. Provide this output and any output or errors received from

         running these commands in the escalation.

--------------------------------------------------------------------------------

-------------------------------------Errors-------------------------------------

Control Station: Check if NBS devices are accessible

Error HC_CS_14505082914: Failed NBS (nd-clnt processes) devices

         access check

         NOTE: Several checks depend on NBS device access to run. These checks

               were not run.

Action :

         1. This may occur if NBS is not configured correctly or if the NBS

            service is not started. Look in the "Checks" section to see if the

            following checks passed:

            * Control Station: Check if NBS configuration exists

            * Control Station: Check if NBS service is started

            * Control Station: Check if NBS clients are started

         If either of those checks did not pass, follow the instructions for

         that check to correct the problem, then rerun nas_checkup to verify

         that the NBS devices can now be accessed.

         2. This may also occur if Blades are powered down or pulled out. If

            you are on the primary Control Station, look in the "Checks"

            section to see if the following check passed:

            * Blades: Check status

         If this check failed, follow its instructions to correct the

         problem, then rerun nas_checkup to verify that the NBS devices can

         now be accessed.

         3. If the problem persists, escalate this issue through your support

            organization. Provide this output and any errors or output that

            occurred running the commands in this procedure in the escalation.

Control Station: Check if NBS service is started

Error HC_CS_14505082915: NBS (nd-clnt processes) service is not (or

         not fully) started

Action :

         Use the command "/sbin/service nbs start" to restart the NBS service

         or reboot the Control Station.

--------------------------------------------------------------------------------

[root@cotys ~]# df -h

Filesystem            Size  Used Avail Use% Mounted on

/dev/hda3             2.0G  1.4G  526M  73% /

none                 1012M     0 1012M   0% /dev/shm

/dev/hda1             259M   12M  237M   5% /boot

/dev/mapper/emc_vg_pri_ide-emc_lv_home

                      591M   17M  544M   3% /home

/dev/mapper/emc_vg_pri_ide-emc_lv_celerra_backup

                      827M  216M  570M  28% /celerra/backup

/dev/mapper/emc_vg_pri_ide-emc_lv_celerra_backendmonitor

                      7.8M  1.2M  6.3M  16% /celerra/backendmonitor

/dev/mapper/emc_vg_pri_ide-emc_lv_celerra_audit

                      117M  5.6M  105M   6% /celerra/audit

/dev/hda5             2.0G  796M  1.1G  42% /nas

6 Posts

December 6th, 2013 08:00

I attached an image from the storage system.

emc.PNG.png

4 Operator

 • 

8.6K Posts

December 6th, 2013 09:00

You won’t get NAS services starting or UniSphere working if the data movers aren’t running

6 Posts

December 6th, 2013 09:00

Hi Rainer:

How does one get the data movers running?

[nasadmin@cotys ~]$ /nas/bin/server_cpu server_2 -reboot now

NAS DB is needed to execute the current command but is unavailable at this time

bg

6 Posts

December 6th, 2013 10:00

I have waited. The last time has been 1 hour. Previously occasions it has been a few hours.

I have verified they have power by looking at them.

yes it was working yesterday.

bg

4 Operator

 • 

8.6K Posts

December 6th, 2013 10:00

Power on and wait

I assume the system was working before

6 Posts

December 6th, 2013 11:00

I just found these..

would this be useful?? in particular i see a backup from "Oct  2 17:02 nasdb_backup.b.tar.gz" which is 2 months before I mucked around.

[root@cotys celerra]# cd /celerra/backup

[root@cotys backup]# ls -lrt

-rw-r--r-- 1 nasadmin nasadmin 16194129 Oct  2 17:02 nasdb_backup.b.tar.gz

-rw-r--r-- 1 nasadmin nasadmin 14571082 Dec  5 06:01 _nasbkup.03.tar.gz

-rw-r--r-- 1 nasadmin nasadmin 14571079 Dec  5 07:01 _nasbkup.04.tar.gz

-rw-r--r-- 1 nasadmin nasadmin 14571087 Dec  5 08:01 _nasbkup.05.tar.gz

-rw-r--r-- 1 nasadmin nasadmin 14571087 Dec  5 09:01 _nasbkup.06.tar.gz

-rw-r--r-- 1 nasadmin nasadmin 14571086 Dec  5 10:01 _nasbkup.07.tar.gz

-rw-r--r-- 1 nasadmin nasadmin 14571084 Dec  5 11:01 _nasbkup.08.tar.gz

-rw-r--r-- 1 nasadmin nasadmin 14571086 Dec  5 12:01 _nasbkup.09.tar.gz

-rw-r--r-- 1 nasadmin nasadmin 14571085 Dec  5 13:01 _nasbkup.10.tar.gz

-rw-r--r-- 1 nasadmin nasadmin 14571094 Dec  5 14:01 _nasbkup.11.tar.gz

-rw-r--r-- 1 nasadmin nasadmin 14571081 Dec  5 15:01 _nasbkup.12.tar.gz

-rw-r--r-- 1 nasadmin nasadmin    70261 Dec  5 16:01 _dbms_backup.01.tar.gz

-rw-r--r-- 1 nasadmin nasadmin 14571597 Dec  5 16:01 _nasbkup.01.tar.gz

-rw-r--r-- 1 nasadmin nasadmin    70264 Dec  5 17:01 _dbms_backup.OK.tar.gz

-rw-r--r-- 1 nasadmin nasadmin    70264 Dec  5 17:01 _dbms_backup.02.tar.gz

-rw-r--r-- 1 nasadmin nasadmin    12475 Dec  5 17:01 stat_groups_backup.tar.gz

-rw-r--r-- 1 nasadmin nasadmin 16235524 Dec  5 17:01 nasdb_backup.1.tar.gz

-rw-r--r-- 1 nasadmin nasadmin 14571580 Dec  5 17:01 _nasbkup.02.tar.gz

4 Operator

 • 

8.6K Posts

December 6th, 2013 12:00

No

I would suggest to engage support

6 Posts

December 9th, 2013 12:00

I have spent some more time trying to figure this out.

I notice that only one of my SP make a login to my host initiator. However what I notice is that

it flips from NO to YES and then back to NO again and then YES and then..

I am not seeing any hardware or software events in the events for the SP.

Well I do see it saying it zeroing out the LUN (a CLI shows that pcrc rebuild/bound is at 100%)

and i see that the channel loop goes up and then goes down.

Are there any other logs or ways I can get more visibility into this.

Does the NAS db have to be operational for a permanent login to occur?

thanks

bg

No Events found!

Top