Files and SQL - Initial Synch & Low Speed Wan

Thanks for the feed back, Tomislav

I have about 60 Gig of File server data to replicate.
Wan is 512Kbps available is bandwidth limited to around 128K
We have a copy of the data from abut 2 weeks ago.
Remote site is 1500 km away so takes a few days to get a copy.
We did a test sync on one directory which took all night but completed successfully.
Subsequent research has indicated using Attribute Compare, as you suggest. And also to break it up into several specs at high level directories rather than a single spec for the whole volume.

When we get this going the next job is try an SQL database from the same site.
I would Quiesce the SQL, take a copy of the database & logs, pre-load that on the target, and then Sync.

What is not 100% clear from the documentation is whether Synchronisation is also a block level operation or whether it will ship the whole file if it has changed.
The description in the Tech Manual suggests it is a sort of block copy but it is not very clear.

How has your experience been on this?

Yes, I have resorted to a similar procedure once. However, bear in mind that files on the source system should not change once you copy them for the target system for this to make sense. That way during the initial sync you will be able to use the "attribute compare only" option for incremental sync, otherwise Replistor will have to go through the complete file to figure out the changes and this may take quite a bit of time on the WAN connection. To be honest, most of the time we resorted to replicating the data over the WAN link anyway, ensuring there is enough space for the Replistor cache, but you mileage may vary depending on the data size and the amount of activity on the database. It would help if you provide us with more details like WAN speed and the DB size...

I have seen a similar method used several times. This does work best if the files are not changing too much.

For prodcution SQL DB's, you will find that you will have to Sync (Inc) them completely in order to ensure the Target is in Sync. Since DB's are always changing this will be required. I suppose the only way to get a round it is to shutdown SQL, copy the data to the device, Sync the data, then start SQL again.

Alternatively, you can bring the Target local to the Source, perform the Sync via LAN, then move it. Allow the Kernel Cache/Logs to caputre the changes while the Target is being moved.

Duncan

Unfortunately I'm pretty certain that it will transfer the complete file over if it is not the same. The only difference between turning on the 'attribute compare only' option and having it off is the method Replistor would use to figure out if the file is different or not. That is way I suggested that the file should not change on the source side. I believe that is not an option for you, but by the time you get to the target site, most of the files on the source will change, meaning that most, if not all files will have to be transferred over to the target site through the WAN anyway.
Also, incremental sync puts additional overhead, and from my experience full sync (depending on the amount and the size of files that need to be transferred) is sometimes faster than incremental one.
But here is an idea. Start the replication to another local machine on the source site. After the initial sync is done, break the link, pull the disk out and take it to the target site. In the mean time the source server will store all the filesystem changes in the Replistor cache. Once in, reestablish the network connection and all the changes will start flowing over and apply on the target server. There are hostname concerns here as obviously the target hostname should stay the same, or possibly you can trick it by changing the DNS entries after the transfer.
The easiest way would be to have the target server physically on the source site, do the initial sync, unplug it and take it over to the target site where you can resume the mirroring operation. This would work nicely, basically instead of just copying the files yourself to the disk, you will use Replistor to do it, and therefore you will have both volumes in sync.

Hope this wasn't too confusing. Let me know if you need me to clarify.

Would it be critical that the Production SQL remains Quiesced until the Target is in sync? Or would we get away with allowing the Source SQL to continue, while we ship the data, and Sync.
I understand that this method would require sufficient KCache & OCxxxx file space to hold the changes, and I guess we would have to do some carefull calculations to ensure that the Sync can catch up in reasonable time.
While we are in test phase we can monitor the Source SQL Cache to see the volume of changes.

You can sync the running SQL servers of course, and you got it right, you need to make sure there will be sufficient space for Data directory...otherwise you will have to do it all over again.

Thanks Guys,
Your input has been very helpful, and I am much more confident about setting up my Replistor.
Hopefully things will go much smoother with the added info

Cheers
JohnG