RM - SQL recovery

Carl, Both of your scenarios are correct.

Scenario 1: This is the procedure and it works well. My SQL Team has tested this many times and the "Recover all databases" they say works great.

Scenario 2: There are no additional steps to "Break Free" from the RM server. It is a good idea (If possible) to disable your schedule. If not just disable it as soon as it's back online.

There is also a command to ensure the drive letters appear on reboot:
C:\mountvol /E to run after the diskpart utility

After that I pass it over to my SQL Team and I've sent them an email to send me over thier steps. As soon as I get them I'll pass the info along

Hi Dave,

Many thank for your replies. Can I just further ask if you do use the metadata in anyway and if you mount every replica as part of the process of creating them? If you do mount the replicas, e.g. for testing purposes, do you mount them using the File System mount option & ¿create and mount a snap of the replica¿?

Many thanks
Carl

Carl
No. The SQL Team just attaches the *.mdf and *.ldf files that are part of the cloned LUN. They do not use any metadata at the DR site or for Local recovery.

I do not mount the LUNs as part of creating them. I have local clones at our production site, I then SAN Copy the clone to the DR site and I have a clone set through Snap View (Not through RM) of the SAN Copied replica. When I test, I use the DR Clone set, not the SAN Copied Replica to preserve a "Gold Copy" at the DR site that still gets updated during testing. This ensures I will not interupt the replication process during testing. I will mount the SAN Copy every other month through RM just to test the mount process but during actual SQL testing I use the Clone.

I have also 3 days of SnapShots just in case corruption or errors get replicated through the process and have mounted and tested them successfully both through RM and manually using the same process.

Let me know if you have any more questions. In my environment, I have restored SQL, which runs our document management system, and the Exchange environment, which interacts with the DMS, successfully and we do this every month.

Thanks,
Dave

Hi Dave,

Many thanks for all your advice. Finally had a chance to implement it and try it out and needless to say worked like magic

I pulled together the below procedure if anybody else requires the necessary steps.

As per always I am inviting comments from the other forum users

Thanks & regards
Carl

In case the production site including the RM server is not available the replica(s) can be recovered manually to the DR SQL Mount Host. The replicas can be mounted read-writable through Navisphere Manager and by modifying the disk attributes using diskpart utilising the below procedure:

1. Assuming that the production SQL server is still down, do all necessary configuration changes to the DR SQL server so that either: it takes on the identity of the production server, or; all users now point to the DR SQL server.
2. It is a good idea (if possible) to disable the RM job schedule. If not just disable it as soon as the Server is back online.
3. Ensure PowerPath on the Mount Host has no unused/failed paths/devices that could cause LUN surfacing issues.
 Ensure there are no dead paths present:
"C:\Program Files\EMC\PowerPath\powermt.exe display dev=all¿
 Remove any unused PowerPath devices:
"C:\Program Files\EMC\PowerPath\powermt.exe check¿
 Save the PowerPath configuration:
"C:\Program Files\EMC\PowerPath\powermt.exe save¿

4. Using RM logs, history tab on the job or email notification decide on the replicas required to recover SQL.
5. Remove the desired replica LUNs from the ¿EMC Replication Storage¿ and ¿EMC RM SANCopy SG¿ Storage Groups.
6. Mount the replicas to the SQL recovery server by adding the database and transaction log LUNs to the mount host¿s storage group.
7. Run a Windows disk rescan by selecting Start > right-click My Computer > Manage > Disk Management > right-click and select ¿Rescan Disks¿.
8. Make a note of the new disk numbers.

Note: You can also verify that you can see these LUNs on the mount host by running a ¿syminq ¿clariion ¿ v¿ command. You can also obtain the drive number by using the syminq command.

9. Use Diskpart to check and if necessary to modify the readonly, hidden and nodefaultdriveletter volume attributes as per below example:

C:\diskpart

DISKPART>list

DISKPART>list disk

DISKPART> select disk 4

DISKPART> detail disk

PowerDevice by PowerPath
Disk ID: 9F0B09CD
Type : FIBRE
Bus : 0
Target : 1
LUN ID : 3

Volume ### Ltr Label Fs Type Size Status Info
---------- --- ----------- ----- ---------- ------- --------- --------
Volume 5 M SQL 2005 MD NTFS Partition 10 GB Healthy
Verify the Disk ID is set correctly. Note the Volume number as well and use it in the next command:

DISKPART> list volume

DISKPART> select volume 5

DISKPART> list partition

DISKPART> select partition 1

DISKPART> detail partition

DISKPART> detail volume

Disk ### Status Size Free Dyn Gpt
-------- ---------- ------- ------- --- ---
* Disk 4 Online 10 GB 0 B

Readonly : No
Hidden : No
No Default Drive Letter: No
Shadow Copy : No

If any of the ReadOnly, Hidden, or No Default Drive Letter are set to yes, clear them with the follow command:

DISKPART> att vol clear readonly hidden nodefaultdriveletter

Volume attributes cleared successfully.

DISKPART>exit


10. To turn on Automount to enable the automatic mounting of new volumes, type the ¿mountvol /E¿ command at a command prompt, and then press ENTER.
11. Assign desired drive letters, right-click on the partition in Windows Disk Management and selecting ¿change Drive Letter and Paths¿.
12. Reboot the server.
13. Start SQL.
14. Launch Enterprise Manager/Management Studio/Query Analyzer and attach the databases and logs.
15. The replicated SQL data should now be on the DR host.

Note: Additional database security information might have to be reapplied.

Note2: If the Navisphere domain master role is defined on the production CLARiiON storage array and is unavailable then Domain/security information for the DR CLARiiON storage array might have to be recreated as per EMC¿s knowledgebase article emc103955!

Thanks Calle for sharing this info.

Carl,
Thanks as well for sharing your DR process for SQL. A couple of questions for you...

What is your reason for removing the LUNs from the replication Storage Groups? Just a precaution or is that the process you would use in a true DR scenario and not a test of the DR data?

I have found in my testing that steps 10 - 12 can all be done through diskpart without the need for a reboot by using the "rescan disk" commands to get the partitions to show up and the "assign letter=x" command to mount the file system to a drive letter. As long as the host is W2K3 with SP1 or later this works fine. Have you tried this?

Aran

Aran,

Precaution is to disable the schedule so the real reason why I have the replicas removed from the RM SGs is that after the restore the source and target (replica) are completely out of synch but what¿s worse is that the RM hasn¿t tracked the changes at the DR end. So a full SAN Copy needs to be run instead of the ISC. The easiest and fastest way I found to force a full instead of incremental update was to delete the replicas from RM and Navisphere and recreate them in Navi Mgr and repopulate RM with the ¿new¿ LUNs.

If it was a true DR scenario we would have to rebuild the RM server and look at a manual SC job from replica (old target) to original source before starting the replication in the original direction again if that is some thing that was still desired¿

Sorry created the procedure doc afterwards and only really had time to get it done once in the first instance so no time to optimize it
Therefore I fully take onboard your thoughts about improving the procedure for steps 10-12 and will for sure try it out when I get the next opportunity

Many thanks
Carl