Best Practices RSS feed for this section

Using Syncovery 8 with Amazon S3 Inventories

Syncovery 8 can use Amazon S3 Inventories. On the second tab sheet of the Internet Protocol Settings dialog, please choose “Use S3 Inventories”.

To make Amazon generate these Inventories, go to the AWS S3 Management Console and refer to the screenshot below.

Inventory Name: your free choice
Filters: empty, or specify the folder you need
Destination Bucket: must be the same bucket that is listed
Destination Prefix: must be empty

Using Google Nearline with Syncovery

We have tested Google Nearline with Syncovery, and yes, it can be used. Here’s how to make it work.

Choose Amazon S3 as protocol, and specify your bucket like this:

bucketname@storage.googleapis.com

You can get your Access ID and Secret Key via the Interoperability tab sheet of the Cloud Storage Settings page
https://console.cloud.google.com/storage/settings

Please don’t choose unsupported features like Reduced Redundancy, as they will cause errors.

See also
https://cloud.google.com/storage/docs/migrating

Using Multiple Accounts With Same Cloud Storage


When you create several profiles using the same cloud storage, you will notice that they automatically connect to the same account. This affects OAuth cloud storage such as Google Drive, OneDrive, DropBox etc.

However, it is also possible to use different accounts for different profiles.

Each cloud account needs to be identified on the Internet Protocol Settings dialog, using the Account (opt.) field. You could enter the email addresses there or anything else. What you enter does not directly mean anything, it is just a name under which the cloud authentication tokens are saved.

Then when you click the Browse button, an Internet Browser window will appear, asking you to authenticate the user. On this screen, the previous user may appear, so you need to click “Not me” or something like that in order to log the previous user out and log the new one in.

If a profile should be associated with a different account, just change the Account ID to something else, and you will have the chance to re-authenticate it.

If you want to completely re-assign your jobs, you can use the button “Forget Cloud Access Tokens” on the Program Settings dialog, tab sheet “Cloud”.

Google Docs protocol completely replaced with Google Drive

In May 2015, the older Google Docs API was shut down by Google, and it can no longer be used. It has been completely replaced with the newer Google Drive protocol.

To handle the situation, the new versions 7.17 and 7.20 beta 15 of Syncovery have been releaed. The Google Drive protocol can now fulfill all tasks that the obsolete Google Docs protocol could.

The main feature that had been missing before these updates was the conversion of native Google documents. This is now available, so that all existing tasks can be performed with the new Google Drive API.

Previous Google Docs jobs will automatically use document conversion where chosen. New jobs, and existing Google Drive jobs, need to have document conversion enabled if desired. The checkmark is on the second tab sheet of the Internet dialog. Note that the available file types for conversion have been updated on the Google Docs tab sheet.

Once a profile is switched over to the new Google Drive protocol, you will have to authenticate Syncovery in a browser window. The prompt will appear if you run the job manually in attended mode.

If you need to use several different user accounts in different profiles, this is entirely possible! Just make sure that each profile has a different Account ID shown on the Internet dialog. When the authentication page appears, make sure you log out of the current Google account and log back in with the next account and authorize Syncovery.

If you need to re-do the authentication, you can click the button “Forget cloud access tokens” on the Program Settings dialog, tab sheet “Cloud”.

Note that your Google accounts must “Allow users to install Google Drive apps”.

Rsync and Syncovery

Syncovery supports the Rsync protocol by using an rsync.exe command line utility. Here’s how to set it up:

On the Program Settings dialog, you need to specify which rsync.exe to use. There’s also a button to download an rsync for Windows installer.

In the profile, you click on the Internet button for one of the two sides of the sync. Choose Rsync as protocol. You can securely connect via SSH or use rsync directly (unsecured). The unsecured connection can be used within a LAN or VPN, for example.

When using it via SSH, you need to have a client certificate and configure it on the Certificates tab sheet, as well as choose it on the Security tab sheet.

Because of limitations of the rsync.exe command line, some features are not working. For example, Syncovery cannot delete any files via rsync.

Because of rsync limitations, it is often better to use SFTP/SSH. It can do block level copying, too, and also it can get a recursive listing similar to rsync (on Unix/Linux servers). The recursive listing checkmark is on the second tab sheet of the Internet/FTP dialog.

Please see the following page for details about block level copying with SFTP:
Block Level Copying

The limitation with block level copying and SFTP is currently that it does block level for uploads only.

Confirmations missing, deletions disabled and similar warnings

When a job runs in unattended mode, there can be some conditions which have to be confirmed when the profile is configured. For example, file deletions have to be confirmed specially, in addition to configuring the sync operating mode to include deletions.

To give the confirmations, please edit your profile in Advanced Mode, go to the Safety section at the bottom and choose the tab sheet “Unattended Mode”.

For an Exact Mirror profile, you would usually make sure that the top four checkmarks are chosen and that the last one is not chosen (Special Safety Checks). The first four checkmarks allow things like replacing files and deleting files. The last checkmark would impose additional safety checks which can prevent file deletions, so you’ll want to remove that checkmark.

In addition, you should specify a percentage of files which may be deleted during a profile run. You should not set this to 100% unless you expect that nearly all files are regularly going to be deleted. This is a safety percentage. If more files than allowed need to be deleted, the program realizes that something is wrong and disables the deletions completely. This will protect you against rare error conditions like disk errors or network communication errors, which can sometimes cause empty folders to be reported on the source side. Due to the percentage limit, the program will not delete everything from the destination if suddenly the source seems to be empty, or nearly empty.

However, even if you do specify 100% for deletions, the program will never delete all files in order to protect your data. If the source side is empty, it will throw an error and refuse to delete everything from the destination. This should be a good safety measure for most use cases. Some customers, on the other hand, have use cases where deleting all files is a regular, normal requirement. These customers can disable the 100% deletion protection by adding the following line to the [Main] section of the INI file:

UnattendedDeleteAllIsOK_IfDelPercIs100=YES

(Add this only if you need to delete 100% of files in a destination folder in unattended mode!)

Keeping 5-7 Full Backup Copies of Your Data in Separate Folders

There are different ways to keep several versions of your files on your backup storage. Firstly, in the profile under Versioning, you can specify to keep a number of
older versions for each file. Each file will be versioned independently, and the Restore Wizard can restore your newest data, or restore according to a specified target date in the past.

However, some customers prefer to have 5 to 7 complete and independent copies of their data. To achieve this, you can use the variable $WEEKDAY in the destination path. In that case, you would not use anything from the Versioning tab sheet.

This will keep seven days of complete folder copies, if the profile runs on every day of the week. It will cause your backup folders to be named Monday, Tuesday, Wednesday, etc. and the next week’s backup is simply going to re-use the same folder from the week before.

For example, your destination path could be like this:
D:\Daily Backups\$WEEKDAY

Use Exact Mirror Mode to make sure that the backup is always identical even though the folder is re-used. Under Safety->Unattended Mode, allow file deletions as well as overwriting newer files with older ones (which will hardly ever be necessary, but should be chosen).

See also: Variables to use in your sync paths

System Requirements

Syncovery runs on:

Windows
All versions of Windows since Windows XP and Server 2003 are fully supported, including Windows 10 and Server 2016 (with Desktop Experience) and all other versions and editions of Windows that have a graphical user interface. Syncovery is available in 32-bit and 64-bit editions. The 64-bit editions require at least Windows Vista. For older 64-bit Windows versions, please use the 32-bit edition of Syncovery.

For Windows 2000, please download Syncovery version 6 from our additional downloads page.

Macintosh
Syncovery runs on all Intel Macs, and on all Mac OS X versions since 10.4 “Tiger” up to including the latest version (currently 10.12 “Sierra”). An older version 5.73 is also available if you need to run it on older PowerPC Macs. This version also requires at least Mac OS 10.4 “Tiger”.

Linux
The command line Edition SyncoveryCL is available for Linux. It should run on any recent 32-bit or 64-bit Linux distribution for Intel processors. See the Linux download page for details.

How to deal with changing USB drive letters easily

A lot of customers ask how they can define a job so that it always copies to or from a USB drive, even if that drive’s letter changes. For example, on one day, your external hard drive might be drive F: and on another day, Windows changes it to drive G:.

The solution is to change the “Base Path” for the external drive in the profile. Instead of the drive letter, type the volume label (volume name). For example, if your drive is named MyUSBDrive, then you would type MyUSBDrive:\FolderA instead of F:\FolderA.

Amazon Glacier Backup

Since version 6.20, Syncovery can upload files to Amazon Glacier. To set up Glacier as the destination, click on the Internet button for the right-hand side and choose Glacier as the Internet Protocol.

Amazon Glacier is intended for long term archiving and backup with the expectation that most operations are uploads and not downloads. Downloads should be a rare exception, since they can be extremely slow. To download a file, Syncovery needs to initiate a job with Amazon Glacier, and even with small files, it takes at least three hours for Amazon Glacier to retrieve the item. The same is true for file listings. Therefore, Glacier cannot be used for two-way synchronizations. It is intended only for backing up and archiving.

Please be aware that there is a download fee from Amazon if you download more than 5% of your data per month (or something like that). It seems that in some special cases, the download fee can be extreme. If you intend to make a full backup and restore for testing, please read the fee schedule closely, and also read articles such as this one:
Is There a Landmine Hidden in Amazon’s Glacier?


Two methods to use Glacier

  • Choose S3 inside our software, and use a lifecycle rule in S3 to move files to Glacier to reduce costs. See the Amazon Web Services Blog
  • Choose Glacier directly in our software. The rest of this article describes this method.

Because there is no quick way to get a file listing from Glacier, Syncovery maintains a local database with the information of all the files on Glacier. This database is automatically maintained and it is independent from the checkmark “Cache Destination File List”. Therefore, with Glacier, there is no need to use the caching option. The local databases used for Glacier are separate for each vault, but shared among all profiles. When you create a new profile that accesses an existing vault, it will alraedy know the vault contents, if the vault has been used on the same machine.

Glacier also cannot rename items which have been uploaded. When you rename or move files on the local side, they have to be uploaded again.

In version 6.20, the download process is still inefficient if you need to download many files. It will initiate the download job for each file and wait for it to complete, then download the file, and then proceed to the next file. You can specify a number of files to copy in parallel in this fashion (on the Files tab sheet), but the limit is 30. So, you can download up to 30 files in 3 to 5 hours, but downloading 300 files will take 30 to 50 hours. This limitation will be removed in a future update.

In light of this, if there is a chance you will need a complete restore, it may make sense to use “Zip Packages” with many files per zip archive, instead of uploading individual files.

This information will be updated as support for Glacier is optimized and more experiences are made with Amazon’s new service.