This post is more than 5 years old
12 Posts
0
3345
December 30th, 2011 17:00
Boot issue after DART upgrade attempt
Happy (early) New Year. I'm new to EMC, but I've immersed myself in the material over the past few weeks. I've found many answers on these forums and I now find myself reaching out and hoping somebody can help me.
I have three Celerra NS20s with CLARiiON CX3-10 backend storage. To take advantage of newer features, I backed up all of the important data from one and used USM to perfom a DART upgrade from 5.x to 6.x (I don't have the exact version in front of me, but it was an approved direct upgrade path according to the compatability documentation. All health checks were good in the pre-upgrade check). During the upgrade, I ran into an issue that I posted a solution on in this thread. Basically, the system had to be manually restarted (reboot -f), but I made sure everything was okay in the log (/var/tmp/upgrade.log). After rebooting and starting USM again, it picked up where it left off and continued on its happy way. Eventually, it got to the step where it had to upgrade the Control Station Linux. Everything went fine from what I could tell, but eventually USM came back and stated that it was taking longer than expected to reboot the Control Station. I wasn't able to ping the IP at all, so I went to the server room and hooked up a monitor and keyboard to the Control Station to see what was going on. It was attempting to boot, but was stuck on loading initrd.img. I rebooted several times, but it hits this right after GRUB tries to boot. In addition to the package for USM, I had a copy of the DART software on DVD. I booted into this and tried to do a repair, but I'm having the same problem. Can I reload the Control Station Linux from scratch without losing my data? I'm not entirely clear if a fresh load will remove any data or what will need to be reconfigured. What other options do I have? All of my storage is still available, but I can't manage it.
These were set up a while ago by a tech, but they have been moved to new location in a secure government building, so getting a tech out isn't a viable option at this point.
TL;DR - Attempted a DART upgrade and now I'm stuck at boot trying to load initrd.img. A tech coming out is unlikely.
Any input/advise is greatly appreciated. Thank you in advance.
ProfessorChaos1
12 Posts
0
January 11th, 2012 06:00
Well, the upgrade has been sucessful. I believe this document decribes what I was experiencing. Additionally, the process took MUCH longer than the estimated time to upgrade the control station Linux. Since the CS was unresponsive and based off of what I was seeing, I thought the kernel was hosed. I did still have to intervene and manually reboot, so the upgrade process did have something wrong. After a little over an hour, I could finally reach the CS. When I resumed the upgrade, it tried upgrading the CS Linux again, but went by in just a few seconds. I wish EMC had more documentation available to customers for those willing to look into it. I understand why they want certain tasks to be performed by their techs, but sparse documentation is more dangerous if one has only part of the procedure. Thank you to those who provided inputs. I will try to reciprocate as best as I can.
dynamox
9 Legend
•
20.4K Posts
0
December 30th, 2011 18:00
bummer, don't have an air card or tethering on your phone ?
ProfessorChaos1
12 Posts
0
December 30th, 2011 18:00
Wireless (including phones) is a big no-no where I'm at
. I'll be the first to admit that it makes it much more difficult for issues like this. What I think I'll do is to get as much detail as possible so that I can do a WebEx from my car or at home on my laptop. Also, the EMC chat still functions at work, so I'm going to reengage that avenue next week when I'm back in the office. In the meantime, I hope to continue the dialogue here and hope that somebody may have info on this. Thanks again dynamox.
dynamox
9 Legend
•
20.4K Posts
0
December 30th, 2011 18:00
If the box is under active support contract, open a ticket. Some things can be done via WebEx.
ProfessorChaos1
12 Posts
0
December 30th, 2011 18:00
I'll double check our support status (I think we're on the tail end of it), but a WebEx will also not work. I was in a chat with an EMC support rep for something else and we attempted a WebEx, but it wouldn't run on our work systems that have outside connectivity. They have it locked down to the bare minimum (secure location). I appreciate the feedback nontheless. Thanks dynamox!
Since this seems to be related to the Linux portion, I'm hoping some Linux gurus can chime in as well. Also, if anybody knows what the results would be if I did a fresh install of the Control Station Linux.
ProfessorChaos1
12 Posts
0
December 30th, 2011 19:00
Well, since it is a long weekend for me, I've taken the time to do some more research. I found this thread where a user had to have the CS replaced and he inquired about his settings being kept with the new equipment. @dynamox, you replied to that thread and indicated that configurations such as file systems, CIFS servers, etc would remain intact. I realize that my situation is somewhat different and that I won't have a tech on hand, but can anybody provide guidance as to whether or not a reinstall of the control station linux from scratch will muck with any of those settings during the install/configuration stage?
Also from that thread, it seems as though I can get running again and import my previous settings from the NASDB backup. Since I'm pretty certain we weren't copying that to another location, hopefully I will be able to at least access the drive and pull it off, but simply having the CS running again with my shares, etc still available is my goal.
dynamox
9 Legend
•
20.4K Posts
1
December 30th, 2011 20:00
i had a CS go bad on me, everything kept running ok (except for snapsure schedules did not run). We had EMC CE come out and replace the thing, i do not know what he did after he physically replaced it but nothing was lost (except for some scripts that i had on the control station)
Rainer_EMC
4 Operator
•
8.6K Posts
1
December 31st, 2011 04:00
In order to do that you need the procedure for control station replacement which saves/restores some config
I would try to get it fixed with customer service before re-installing though
ProfessorChaos1
12 Posts
0
January 3rd, 2012 08:00
Well, I was planning on tackling this when I got in this morning, but I had some other things come up and I won't be back to work until tomorrow. I know from this thread (and all others) that going through customer service is my best route. I'm am still going to leverage all of my support options through them and see what I can find out. I'm also going to lean on some local guys with more EMC background than me to get their inputs as well.
With that being said, I want to make sure I have a way forward if those fall through. @Rainer_EMC, you mentioned that there are procedures for replacing the control station. While I wouldn't be replacing it, I'm sure it would have information pertinent to my situation. I searched the support site, but I couldn't find anything. Does anybody know if there are instructions available to customers? As much as I'd love to have something like this, it doesn't seem likely that there are instructions for anybody outside of service techs and certain partners.
I think that somebody with more Linux knowledge than me may be able to offer some suggestions based off of the situation. At this point, we're really looking at a Linux issue and the right commands will likely get me to a point where I can boot the kernel. Since I can get to the GRUB menu, I'm going to see if it's checking for the updated kernel or at least find what's causing it to hang up.
ProfessorChaos1
12 Posts
0
January 3rd, 2012 09:00
Also, since I'm already heaping it on
, a quick follow up on an earlier question. I know the control station can be replaced without affecting my shares, etc, but I'm wondering if reinstalling the control station will overwrite anything if I push forward with that route as a last ditch. It give me the option to to a destructive install of the control station Linux, but I'm not sure if that "destructive" part is only the control station itself that will have to be reconfigured, or if it will clear out everything. I know the routine by now
, so I will ask EMC support this same question. I'm just looking for guidance here as well.
Thank you for bearing with me. I really do appreciate those of you taking your own time to help me with something out of your own generosity. I can only offer you my thanks and try to reciprocate by trying to help others on here.
Rainer_EMC
4 Operator
•
8.6K Posts
0
January 4th, 2012 04:00
Don't do it yourself - destructive is what it says
Let EMC support solve it with the proper procedures
Rainer
Rainer_EMC
4 Operator
•
8.6K Posts
0
January 11th, 2012 09:00
FYI,
this has changed with newer products.
The “older” Celerra systems were customer installable (i.e. configuring the pre-installed OS) but upgrade was a service activity.
With the VNX series the systems are customer upgradeable and a GUI (UniSphere Service Manager) while do the full process from downloading the software, running the pre-upgrade health check to installing and rebooting.
You can see a video demo about it in action on Powerlink
http://powerlink.emc.com/km/live1/en_US/Offering_Basics/Demo/VNX_Video_Upgrading_VNX_Operating_Environment.mp4
Regards
Rainer
ProfessorChaos1
12 Posts
0
January 12th, 2012 17:00
I'm getting access denied...probably since I don't have any VMX series registered to me.
I know the Celerra is older, but EMC allows the upgrade to 6.x from certain versions of 5.x by the customer. As you mentioned, I used USM to perform the upgrade and had all of the benefits of the GUI. Only when I had problems did I have to turn to the CLI. It's buried in the documentation, but there are special steps if you are using USM to upgrade systems that aren't able to connect to PowerLink. Basically, you have to install USM on a system with PowerLink access, enter your system details, download the upg package, burn it to disc, and sneaker-net it to the system. For some reason, USM wouldn't find the package in the download folder when I dropped it there, so I just moved it and browsed to the new location. I mention this mainly because the techs in the chat support had no idea how to do this and couldn't understand how I could have a system without outside connectivity
. That's what happens when you outsource your tech support! Thanks again for the help. Anybody can feel free to ask me questions about my experience if you are in a similar situation.
dynamox
9 Legend
•
20.4K Posts
0
January 12th, 2012 20:00
no worries, the same person who just asked those Celerra os basics questions will be assisting you in chat tomorrow