Page 1 of 2

Recovery from S3

Posted: Fri Nov 27, 2020 3:03 am
by steinitz
Hello Tobias and friends

I got the backup to s3 working, partly from information provided in this thread and partly from some helpful troubleshooting from Tobias. Thanks for that.

Now, I'd like to test a recovery to a fresh server. Remember my backup target is /. Would you have any advice on settings for the recovery before I go ahead?

I note that searching the forum (and the main site) for recovery yields little. Is anyone testing recovery? :D

Cheers,
Steve

Re: Recovery from S3

Posted: Fri Nov 27, 2020 2:43 pm
by tobias
Hello,
I think you're the first customer to try a recovery to the root folder, even if some subfolders are excluded. This is a bit of new territory for Syncovery. It should theoretically work but all of the testing goes into just backing up and restoring data folders, not a complete Linux file system. I will test this here myself too as soon as possible. I don't know what's going to happen when you restore right into the running Linux system. We will see.

The restore job should basically be a renamed copy of the backup job with the same settings, except copying from right to left.

Re: Recovery from S3

Posted: Sun Nov 29, 2020 12:34 am
by steinitz
Thanks for your reply, Tobias, and for your offer to experiment with restoring a complete linux file system.

I like your idea of making a renamed copy for restore. But since it will run on another server, I'd need to export/import it. I don't see a way to do that in the web UI. Can you help with that?

Re: Recovery from S3

Posted: Sun Nov 29, 2020 10:57 pm
by tobias
Hello,
you can export and import via the context menu. Just right-click on any profile to get the menu. You could download the complete config file (which includes passwords), or export in XML format (which doesn't include the S3 access credentials).

You can import via the same menu.

Re: Recovery from S3

Posted: Mon Nov 30, 2020 7:39 pm
by tobias
I am testing the restore now, and also about to complete the next Syncovery update for Linux.

There seems to be an issue with symlinks which I am looking into.

Re: Recovery from S3

Posted: Tue Dec 01, 2020 11:19 pm
by tobias
Hello,
I am about to release version 9.23 with some issues fixed.

My first restore test resulted in the X Window GUI not starting up - does your server even have that? Maybe my problem is different hardware in the VM where I am testing the restore. The original machine that I took the backup from is a physical machine.

I am trying to make a list of folders and files which you should definitely not restore from a different server. So far I have:
/boot
/etc/fstab
/etc/hostname

probably also:
/etc/passwd
/etc/shadow*

These need to be put under Masks & Filters -> Exclusions, with the radio button set to "Ignore Totally".

And of course, I am assuming that these exclusions were already in place when the backup was made:
/dev;/proc;/mnt;/var;/run;/tmp;/sys

Re: Recovery from S3

Posted: Wed Dec 02, 2020 8:48 am
by tobias
I have made a successful restore over a new Linux installation now. I will release Syncovery 9.23 today, which you should use to do the restore.

I have added a lot more exclusions, which were easily found by first running the job "Without Deletions" and then again to see what Syncovery wanted to delete. For example, I had a different Grub version on the VM (a legacy non-EFI version I believe). I didn't want that deleted.

The Restore job needs to have all the Exclusions that the original backup job had, so it won't try to delete items in folders like
/dev;/proc;/mnt;/var;/run;/tmp;/sys

These are my additional exclusions:
/run/udev
/swapfile
/etc/X11
/etc/xdg

Plus, grub-related:
/usr/bin/grub-ntldr-img
/usr/lib/grub-legacy
/usr/sbin/*grub*
/usr/share/bug/grub-pc
/usr/share/bug/grub-pc-bin
/usr/share/grub-gfxpayload-lists
grub-pc-bin

And some more personal ones:
.cache
/home/tobias/.local/share/Trash

Re: Recovery from S3

Posted: Wed Dec 02, 2020 11:19 am
by steinitz
Great news Tobias. Thank you.

If this works it's a very specialized and useful type of full-server backup / restore - not available, as far as I know, with any other backup tool. Bravo

I'll try a restore, per your instructions, in the morning, and report back.

Re: Recovery from S3

Posted: Wed Dec 02, 2020 11:23 am
by tobias
Hi,
great, just wait for 9.23 ...

Re: Recovery from S3

Posted: Wed Dec 02, 2020 9:45 pm
by tobias
Hello,
I have released version 9.23 now with fixes restoring permissions and symlinks from a cloud storage.

You may also want to put .Syncovery into the Exclusion Masks so that Syncovery won't replace its own config file from the backup.

You can either import the profile and change the copying direction, or create a new one with these settings:

To restore, copy right to left.

Use Exact Mirror, but let the first run go without deletions. This can be done when you run it "With Preview".

Case Sensitive Mode (very important)

Masks & Filters->Exclusions: /boot;/etc/fstab and others
The Exclusions must also contain all exclusions from the original backup file and be set to "Ignore Totally".

General Filters: use $symlink files for right side

Special->Permissions: Copy both from source, compare all, use metadata files for right side

Special->More->Extended Attributes: copy, compare, use metadata files for right side

Comparison->More: I did not choose Folder Timestamps (might not work with S3 anyway, this is optional, takes some additional time).