This post is more than 5 years old
153 Posts
0
5525
September 15th, 2015 01:00
Root2Root migration in a Av /DD setup
Hi all,
I'm testing a Root2Root replication in an Avamar / Data Domain setup and it doesn't seem that the backpus that reside
on the Data Domain is being replicated. This is a fresh install of Avamar and just used for testing
Source site:
Avamar v. 7.1.1-145
DDOS v. 5.5.0.4
Dest site:
Avamar v. 7.1.2-21
DDOS v. 5.5.0.8
This is the command that I'm using and I can see that everything that resides on Avamar is being replicated without problems
nohup avrepl --operation=replicate --[replscript]dstaddr=lpa01.emcsw.loc --[replscript]dstid=root --dstpassword=xxxxxx --[avtar]id=root --[replscript]fullcopy=true --ap=xxxxxx --send-adhoc-request=false --max-streams=8 --[replscript]timeout=0
This is what I see in the nohup.out log
avtar Error <0000>: Replication failed - id: 2, unexpected exception caught: ddr_replthread:2:replicateSlice: ddp_filecopy_status failed
[25068] [140419191834368] Tue Sep 15 09:45:44 2015
ddp_filecopy_status() failed, start_offset[0], length[20971520], Err: 5009-filecopy operation failed (nfs:I/O error)
[25068] [140419187623680] Tue Sep 15 09:45:43 2015
ddp_filecopy_status() failed, start_offset[0], length[20971520], Err: 5009-filecopy operation failed (nfs:I/O error)
[25068] [140419188676352] Tue Sep 15 09:45:43 2015
ddp_filecopy_status() failed, start_offset[0], length[20971520], Err: 5009-filecopy operation failed (nfs:I/O error)
[25068] [140419189729024] Tue Sep 15 09:45:43 2015
ddp_filecopy_status() failed, start_offset[0], length[20971520], Err: 5009-filecopy operation failed (nfs:I/O error)
[25068] [140419192887040] Tue Sep 15 09:45:43 2015
ddp_filecopy_status() failed, start_offset[0], length[20971520], Err: 5009-filecopy operation failed (nfs:I/O error)
[25068] [140419190781696] Tue Sep 15 09:45:42 2015
ddp_filecopy_status() failed, start_offset[0], length[20971520], Err: 5009-filecopy operation failed (nfs:I/O error)
[25068] [140419187623680] Tue Sep 15 09:44:22 2015
ddp_filecopy_stop() failed, Err: 5004-nfs filecopy stop failed (nfs: No such file or directory)
[25068] [140419192887040] Tue Sep 15 09:44:22 2015
ddp_filecopy_stop() failed, Err: 5004-nfs filecopy stop failed (nfs: No such file or directory)
[25068] [140419190781696] Tue Sep 15 09:44:22 2015
ddp_filecopy_stop() failed, Err: 5004-nfs filecopy stop failed (nfs: No such file or directory)
[25068] [140419188676352] Tue Sep 15 09:44:22 2015
ddp_filecopy_stop() failed, Err: 5004-nfs filecopy stop failed (nfs: No such file or directory)
avtar Error <18797>: Replication failed - The replicator returned an error.
avtar Error <18773>: MReplication failed -- backup: 1D0ECA656E313A2, from dd02.emcsw.loc(1):avamar-1441900151 to dd04.emcsw.loc(1):avamar-1442217989, DDR result code: 4912, desc: No error description available.
avtar Warning <18125>: Calling DDR_REPLICATE returned result code:5009 message:I/O error
avtar Error <10612>: Replication failed -- could not replicate file /cur/942c32b70715f427d2988ae8e86cc2c41e14496a/1D0ECA656E313A2/ddr_files.xml to /STAGING/942c32b70715f427d2988ae8e86cc2c41e14496a/BACKUP-5984104540515F95695C530B1C6CD980F3F7B51C-1D0ECA656E313A2/ddr_files.xml, LSU: avamar-1442217989, DDR result code: 5009, desc: I/O error
avtar FATAL <12527>: Replication failed
avtar FATAL <40009>: DDR encountered errors.
They way I read the output from the log is that it doesn't seem like src site can send data to dst site ddboost device in Data Domain. The src site is of course created with a DD Boost device and some backup has been running to
DD and some directly to Avamar.
The dst site is created with a DD boost device but is an "empty".
I'm pretty sure that you should be able to do a Root2Root in a combined enviroment but do the DDboost device need to be the same name on both src & dst? Do you need to rename the dst DD system to src DD name?
Any feedback is much appreciated
broeste1
153 Posts
1
September 16th, 2015 05:00
Hi All,
I solve the issue. I'm afraid that is was as simple as a missing Replicator license on the dst Data Domain. Should have checked that as one of the first things but you learn by your mistakes :-)
After I added the Replicator license the replication starts with both data on Avamar and Data Domain.
I still see an issue with the MCS needs to be started on the dst site before any replication can start. Accordingly to the guide the MCS should be stopped but then I get the above error message
umichklewis
3 Apprentice
•
1.2K Posts
0
September 15th, 2015 05:00
The DD system names do not need to match. Can you confirm the Avamar nodes on the destination side are intergrated with the destination side DD? You should be able to see the DD array on the destination side in the Avamar GUI on the destination side. I would check to make sure the DDBoost passwords are set on each device, and that a network route exists between the two DD arrays (you can check from the CLI of each DD).
Let us know if that helps!
broeste1
153 Posts
0
September 15th, 2015 14:00
Nope, didn't help. Everything is running on the same subnet in my test enviroment and no Firewall in between src and dst.
The dst DD system is integrated with the dst Avamar and everything looks fine in both GUI and CLI.
I'm following this guide for the migration EMC® Avamar® 7.0 System Migration Using Root-to-Root Replication. Accordingly to this guide the MCS has to be stopped on the src system. Not sure if this is related but when I stop the MCS on dst and run the avrepl command i get the following in the
2015/09/15-21:10:02.44024 [avagent] Debug output redirected
2015-09-15 23:10:02 avagent Info <5008>: Logging to /usr/local/avamar/var/client/NAH-1442351402392-1008-replicator-avagent.log
2015/09/15-21:10:02.44035 [avagent] Config: VARDIR=/usr/local/avamar/var, HOMEDIR=/root
2015/09/15-21:10:02.44038 [avagent] Looking for flag file "/usr/local/avamar/var/avamar.cmd"
2015/09/15-21:10:02.44040 [avagent] Looking for flag file "/usr/local/avamar/var/avagent.cmd"
2015-09-15 23:10:02 avagent Info <19803>: Ignoring the --service flag.
2015-09-15 23:10:02 avagent Info <5702>: Command Line: /usr/local/avamar/bin/avagent.bin --gencerts="true" --mcsaddr="lpa01.emcsw.loc" --mcsport="28001" --conntimeout="120" --logfile="/usr/local/avamar/var/client/NAH-1442351402392-1008-replicator-avagent.log" --debug="false"
2015-09-15 23:10:02 avagent Info <5703>: Parsed Flags: /usr/local/avamar/bin/avagent.bin --gencerts=true --mcsaddr=lpa01.emcsw.loc --mcsport=28001 --conntimeout=120 --logfile=/usr/local/avamar/var/client/NAH-1442351402392-1008-replicator-avagent.log --debug=false
2015-09-15 23:10:02 avagent Info <19807>: Creating certificates in '/usr/local/avamar/etc/172.30.17.116'
2015-09-15 23:10:02 avagent Info <18918>: Registration: Processing secure registration with the MCS.
2015-09-15 23:10:02 avagent Info <18921>: Registration: Requesting root CA from the MCS.
2015-09-15 23:10:02 avagent Error <5365>: Cannot connect to 172.30.17.116:28001.
2015-09-15 23:10:02 avagent Info <5059>: unable to connect, sleep(60) then retrying
2015-09-15 23:11:02 avagent Error <5365>: Cannot connect to 172.30.17.116:28001.
2015-09-15 23:11:02 avagent Info <5059>: unable to connect, sleep(60) then retrying
2015-09-15 23:12:02 avagent Error <5365>: Cannot connect to 172.30.17.116:28001.
2015-09-15 23:12:02 avagent Info <5059>: unable to connect, sleep(60) then retrying
2015-09-15 23:13:02 avagent Error <5365>: Cannot connect to 172.30.17.116:28001.
2015-09-15 23:13:02 avagent Info <5059>: unable to connect, sleep(60) then retrying
2015-09-15 23:14:02 avagent Error <5365>: Cannot connect to 172.30.17.116:28001.
2015-09-15 23:14:02 avagent Info <5059>: unable to connect, sleep(60) then retrying
Shouldn't the MCS be stopped on the dst system?