remote listing service not being triggered

English Support for Syncovery on Linux etc.
Post Reply
patanne
Posts: 12
Joined: Wed Mar 10, 2021 2:51 pm

remote listing service not being triggered

Post by patanne »

I am trying to perform a pull backup and the job, for some reason, is not triggering the remote listing service on the machine with the files to be backed up.

I have a MacOS backup server running Syncovery 9.32a. I am trying to perform a pull backup from a machine running Debian Buster and the Syncovery Remote service 9.03 build 4. The job has write access to the root backup folder. The SyncoveryRS service is running and is monitoring the folder. When the job starts it is creating the Syncovery.*request*$$$ file in the root folder of the backup job. See below. But, for whatever reason, the SyncoveryRS service never engages and collects the file list. I watch with top the whole time as well as with the service web page (port 8948) and nothing changes. It's as if the service does not see the file.

Some things/assumptions to note:
  • Without remote listing service turned on the job runs fine and has been for about 1.5 years.
  • Since I do not know how you are watching the directory it should be known that there are roughly 1.2 million files and directories being watched by syncthing on the same machine so I have had to bump up /proc/sys/fs/inotify/max_user_watches to 524288. Syncthing is pretty good about telling you if the max_user_watches count is too low, so the assumption is all is well.
    Since this SyncoveryRS service runs as root, I am assuming there are no ACL issues. To be thorough there is a screenshot below of rights on a listing file after creation.
Any idea how to track this one down?


The listing request file has been properly created.
2021-03-20_14-29-58.png
2021-03-20_14-29-58.png (138.44 KiB) Viewed 5843 times

ACL's on the file.
2021-03-20_15-05-17.png
2021-03-20_15-05-17.png (42.27 KiB) Viewed 5843 times

The service watching the correct directory but not processing request.
2021-03-20_14-46-57.png
2021-03-20_14-46-57.png (35.67 KiB) Viewed 5843 times

tobias
Posts: 1589
Joined: Tue Mar 31, 2020 7:37 pm

Re: remote listing service not being triggered

Post by tobias »

I would still assume some permissions thing, even root can be blocked by SELinux or other security layers.

The Remote Service does not use any file system event watches, it just regularly lists the files in the folder.

Can you do a "cat" command on the filename and show the output?

patanne
Posts: 12
Joined: Wed Mar 10, 2021 2:51 pm

Re: remote listing service not being triggered

Post by patanne »

2021-03-20_15-04-44.png
2021-03-20_15-04-44.png (43.34 KiB) Viewed 5835 times
SELinux is not being used here.

Also, I was mistaken about the distro. It is Debian Stretch, not Buster.
Last edited by patanne on Sun Mar 21, 2021 3:05 pm, edited 1 time in total.

tobias
Posts: 1589
Joined: Tue Mar 31, 2020 7:37 pm

Re: remote listing service not being triggered

Post by tobias »

Hello,
does restarting the Remote Service help at all?

Or could you maybe stop it and run it in a terminal so you can see some of its console output?

You you enter:
SyncoveryRS stop
to stop it, and make sure it's stopped, and then

SyncoveryRS

to run it with console output.

patanne
Posts: 12
Joined: Wed Mar 10, 2021 2:51 pm

Re: remote listing service not being triggered

Post by patanne »

I figured it out.

When you install the service it installs syncoveryrs.service under systemd. I forget if I launched the service manually or it started as part of the installation. Nevertheless, when I then followed the instructions and launched the background process (SyncoveryRS start & SyncoveryRS stop) that became the responding service for the web server on either 8949 or 8948. I then had a situation like this, where the one dated today was the systemd process I restarted and the ones from March 20 were the background processes...

Code: Select all

root@ns-6848:~# ps aux | grep Syncovery | grep -v grep
root      6455  0.0  0.0 307784  7676 ?        Sl   11:49   0:00 /usr/lib/SyncoveryRS/SyncoveryRS start
root      6890  0.0  0.0 375936  5540 ?        Sl   Mar20   0:40 /usr/lib/SyncoveryRS/SyncoveryRS start
root      7331  0.0  0.0 306840  6056 ?        Sl   Mar20   0:39 /usr/lib/SyncoveryRS/SyncoveryRS start
root      7625  0.0  0.0 307816  5464 ?        Sl   Mar20   0:38 /usr/lib/SyncoveryRS/SyncoveryRS start
root     18859  0.0  0.0 307816  5576 ?        Sl   Mar20   0:37 /usr/lib/SyncoveryRS/SyncoveryRS start
By issuing the following I killed all conflicting processes...

Code: Select all

killall -r SyncoveryRS
systemctl restart syncoveryrs.service  # restart to clear out residue
and now have this...

Code: Select all

root@ns-6848:~# ps aux | grep Syncovery | grep -v grep
root      6722  0.0  0.1 306808  9400 ?        Sl   12:06   0:00 /usr/lib/SyncoveryRS/SyncoveryRS start
The background process config file is in /<<username>>/.Syncovery/SyncoveryRemoteService.xml
The systemd config file is in /.Syncovery/SyncoveryRemoteService.xml

Here is what I cannot explain.
  • When all processes were running, the background process took over the web interface on port 8948. At that time port 8949 was not responsive.
  • I recall starting the background process with the NODUPE option. Not sure why I still got the issue above.
  • Even though the background process was updating the config file it was not monitoring folders. Does it need to be restarted to reload the config?
  • The systemd process was monitoring the folders but was looking at an empty config file since the web interface was modifying the one for the background process.
  • Once the systemd process was running port 8949 became the operational one.

tobias
Posts: 1589
Joined: Tue Mar 31, 2020 7:37 pm

Re: remote listing service not being triggered

Post by tobias »

Hi,
OK, interesting!

I can't explain everything either, but I need to look into NODUPE.

Originally NODUPE was only added because we didn't have a proper installation package yet and the service couldn't daemonize properly yet.

The "start" parameter makes the process fork and become a daemon. The documentation was properly updated incorrectly, maybe NODUPE isn't actually supported.

In any case, multiple parallel processes aren't supported. One may deactivate another. Additional processes may still stay in memory but don't do anything.

Post Reply