Recovery from S3

English Support for Cloud Storage
steinitz
Posts: 13
Joined: Wed Sep 16, 2020 2:54 am

Recovery from S3

Post by steinitz »

Hello Tobias and friends

I got the backup to s3 working, partly from information provided in this thread and partly from some helpful troubleshooting from Tobias. Thanks for that.

Now, I'd like to test a recovery to a fresh server. Remember my backup target is /. Would you have any advice on settings for the recovery before I go ahead?

I note that searching the forum (and the main site) for recovery yields little. Is anyone testing recovery? :D

Cheers,
Steve

tobias
Posts: 1604
Joined: Tue Mar 31, 2020 7:37 pm

Re: Recovery from S3

Post by tobias »

Hello,
I think you're the first customer to try a recovery to the root folder, even if some subfolders are excluded. This is a bit of new territory for Syncovery. It should theoretically work but all of the testing goes into just backing up and restoring data folders, not a complete Linux file system. I will test this here myself too as soon as possible. I don't know what's going to happen when you restore right into the running Linux system. We will see.

The restore job should basically be a renamed copy of the backup job with the same settings, except copying from right to left.

steinitz
Posts: 13
Joined: Wed Sep 16, 2020 2:54 am

Re: Recovery from S3

Post by steinitz »

Thanks for your reply, Tobias, and for your offer to experiment with restoring a complete linux file system.

I like your idea of making a renamed copy for restore. But since it will run on another server, I'd need to export/import it. I don't see a way to do that in the web UI. Can you help with that?

tobias
Posts: 1604
Joined: Tue Mar 31, 2020 7:37 pm

Re: Recovery from S3

Post by tobias »

Hello,
you can export and import via the context menu. Just right-click on any profile to get the menu. You could download the complete config file (which includes passwords), or export in XML format (which doesn't include the S3 access credentials).

You can import via the same menu.

tobias
Posts: 1604
Joined: Tue Mar 31, 2020 7:37 pm

Re: Recovery from S3

Post by tobias »

I am testing the restore now, and also about to complete the next Syncovery update for Linux.

There seems to be an issue with symlinks which I am looking into.

tobias
Posts: 1604
Joined: Tue Mar 31, 2020 7:37 pm

Re: Recovery from S3

Post by tobias »

Hello,
I am about to release version 9.23 with some issues fixed.

My first restore test resulted in the X Window GUI not starting up - does your server even have that? Maybe my problem is different hardware in the VM where I am testing the restore. The original machine that I took the backup from is a physical machine.

I am trying to make a list of folders and files which you should definitely not restore from a different server. So far I have:
/boot
/etc/fstab
/etc/hostname

probably also:
/etc/passwd
/etc/shadow*

These need to be put under Masks & Filters -> Exclusions, with the radio button set to "Ignore Totally".

And of course, I am assuming that these exclusions were already in place when the backup was made:
/dev;/proc;/mnt;/var;/run;/tmp;/sys

tobias
Posts: 1604
Joined: Tue Mar 31, 2020 7:37 pm

Re: Recovery from S3

Post by tobias »

I have made a successful restore over a new Linux installation now. I will release Syncovery 9.23 today, which you should use to do the restore.

I have added a lot more exclusions, which were easily found by first running the job "Without Deletions" and then again to see what Syncovery wanted to delete. For example, I had a different Grub version on the VM (a legacy non-EFI version I believe). I didn't want that deleted.

The Restore job needs to have all the Exclusions that the original backup job had, so it won't try to delete items in folders like
/dev;/proc;/mnt;/var;/run;/tmp;/sys

These are my additional exclusions:
/run/udev
/swapfile
/etc/X11
/etc/xdg

Plus, grub-related:
/usr/bin/grub-ntldr-img
/usr/lib/grub-legacy
/usr/sbin/*grub*
/usr/share/bug/grub-pc
/usr/share/bug/grub-pc-bin
/usr/share/grub-gfxpayload-lists
grub-pc-bin

And some more personal ones:
.cache
/home/tobias/.local/share/Trash

steinitz
Posts: 13
Joined: Wed Sep 16, 2020 2:54 am

Re: Recovery from S3

Post by steinitz »

Great news Tobias. Thank you.

If this works it's a very specialized and useful type of full-server backup / restore - not available, as far as I know, with any other backup tool. Bravo

I'll try a restore, per your instructions, in the morning, and report back.

tobias
Posts: 1604
Joined: Tue Mar 31, 2020 7:37 pm

Re: Recovery from S3

Post by tobias »

Hi,
great, just wait for 9.23 ...

tobias
Posts: 1604
Joined: Tue Mar 31, 2020 7:37 pm

Re: Recovery from S3

Post by tobias »

Hello,
I have released version 9.23 now with fixes restoring permissions and symlinks from a cloud storage.

You may also want to put .Syncovery into the Exclusion Masks so that Syncovery won't replace its own config file from the backup.

You can either import the profile and change the copying direction, or create a new one with these settings:

To restore, copy right to left.

Use Exact Mirror, but let the first run go without deletions. This can be done when you run it "With Preview".

Case Sensitive Mode (very important)

Masks & Filters->Exclusions: /boot;/etc/fstab and others
The Exclusions must also contain all exclusions from the original backup file and be set to "Ignore Totally".

General Filters: use $symlink files for right side

Special->Permissions: Copy both from source, compare all, use metadata files for right side

Special->More->Extended Attributes: copy, compare, use metadata files for right side

Comparison->More: I did not choose Folder Timestamps (might not work with S3 anyway, this is optional, takes some additional time).

Post Reply