Problems with dual destinations

English Support for Syncovery on Linux etc.
Post Reply
Jean-Michel
Posts: 37
Joined: Fri Jun 10, 2022 2:12 pm

Problems with dual destinations

Post by Jean-Michel »

Hello
I have difficulties to setup a new platform on a Debian using Syncovery 10.18.4 to achieve the following
- Scan a local path for files
- For each file, move to 2 destinations
- Primary is a S3 destination (temporary path, it will then move by a cloud process)
- Secondary is another local path
Multiple destination is set as "Compare only against main destination, Copy to all destinations"

So at the end each file should have been copied to S3, copied to the other local path and deleted from the original local path

I have inconsistent behavior depending if I run the profile as scheduled or attended with preview
Sometime it seems to work
Sometimes the copy to the local path is ok but the upload to S3 fails

Here below is an exemple of error messages I have in the logs.

Thanks for your advices

Code: Select all

Syncovery v10.18.4, build 284 (64-bit CL)

Licensed to: unregistered
Running on : Store04
Running as : root
System Info: Linux Store04 6.1.0-35-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.137-1 (2025-05-07) x86_64 GNU/Linux

Log for profile TEST, started at 14:29:57 on 10/07/2025

LEFT: /data/TEST
RIGHT: S3://name-of-the-bucket/watchfolder/_to_transcode

The profile is run in the background with Sync Preview

Previous run was at 10/07/2025 14:26:34: INCOMPLETE: 0 copied of 113, Errors with 3 File(s), Duration: 00:00:22


Connecting with name-of-the-bucket
Working with folder emulation mode for S3.
Using OpenSSL 3.0.16 11 Feb 2025: libssl.so.3, libcrypto.so.3

TLS Certificate verified successfully.
Copying Direction : Left To Right

Case Sensitive Mode is ON.
Scanning Folders Started At 10/07/2025 14:30:04
Getting S3 Listing (splitting after 2000000 entries)
S3 Listing Downloading Time: 0.12 seconds for 1 HTTP requests
S3 Listing Processing Time : 0.00 seconds for 2 directory entries

Free Space on left side :    50,194 GB, needed:    0 Bytes
Free Space on right side:  Not checked, needed:   1,488 GB

Main Copying Phase Started At 10/07/2025 14:30:36

Now copying to additional destination: /data/DONE/BE/Q S01 2016-2017/160921 BE QUOTIDIEN CLEAN.mxf
Using Internal Copying Function
MOVE /data/TEST/BE/Q S01 2016-2017/160921 BE QUOTIDIEN CLEAN.mxf  (14.1GB)

Still Working At 10/07/2025 14:31:06, Execution Phase: Main Action (Copying etc.) (CopyFiles)
Still Working At 10/07/2025 14:31:36, Execution Phase: Main Action (Copying etc.) (CopyFiles)
Now copying to additional destination: /data/DONE/BE/Q S01 2016-2017/160922 BE QUOTIDIEN CLEAN.mxf
MOVE /data/SOURCE/BE/Q S01 2016-2017/160922 BE SHOWNAME CLEAN.mxf  (17.9GB)

Now copying to main destination: S3://name-of-the-bucket/watchfolder/_to_transcode/BE/Q S01 2016-2017/160926 BE CLEAN.mxf
Unix error code 2 (File not found) creating S3://name-of-the-bucket/watchfolder/_to_transcode/BE/Q S01 2016-2017/160926 BE CLEAN.mxf
CreateFile S3://name-of-the-bucket/watchfolder/_to_transcode/BE/Q S01 2016-2017/160926 BE CLEAN.mxf, failed with error 2: File not found
Unix error code 2 (File not found) creating S3://name-of-the-bucket/watchfolder/_to_transcode/BE/Q S01 2016-2017/160926 BE CLEAN.mxf
CreateFile S3://name-of-the-bucket/watchfolder/_to_transcode/BE/Q S01 2016-2017/160926 BE CLEAN.mxf, failed with error 2: File not found
Creation error: S3://name-of-the-bucket/watchfolder/_to_transcode/BE/Q S01 2016-2017/160926 BE CLEAN.mxf (File not found)
14:31:53.707: canceling copying loop (C)
Now copying to additional destination: /data/DONE/BE/Q S01 2016-2017/160926 BE CLEAN.mxf
User abort during file /data/DONE/BE/Q S01 2016-2017/160926 BE CLEAN.mxf
Incomplete file deleted from destination
Now copying to main destination: S3://name-of-the-bucket/watchfolder/_to_transcode/BE/Q S01 2016-2017/160921 BE SHOWNAME CLEAN.mxf
Uploading to 160921 BE SHOWNAME CLEAN.mxf with chunk size of 67108864 Bytes and 3 Threads. File Size:    14 GB
sslshutdown Error Message: 167772287 error:0A00007F:SSL routines::bad write retry
Line 1191 in ../ssl/record/rec_layer_s3.c: ssl3_write_pending; , 1
sslshutdown Error Message: 167772287 error:0A00007F:SSL routines::bad write retry
Line 1191 in ../ssl/record/rec_layer_s3.c: ssl3_write_pending; , 1
sslshutdown Error Message: 167772287 error:0A00007F:SSL routines::bad write retry
Line 1191 in ../ssl/record/rec_layer_s3.c: ssl3_write_pending; , 1
Exception with multipart upload: Operation Canceled, CP: S3PutMPEX:Operation Canceled
Now deleting the incomplete upload: watchfolder/_to_transcode/BE/Q%20S01%202016-2017/160921%20BE%20SHOWNAME%20CLEAN.mxf?uploadId=xbEkwN8ogW05OewLINGXZCigwwxH6yvRXx2vVqPOB_yUs86rluMxfgOLXIvKMwokXhdUOHDdsOp9fAWP38M7fmBpl3yGVuljRJz1q9SOJKALMFEbjs2XSDlYjgpVpr7s
10/07/2025 14:31:53 raising Synapse exception Synapse TCP/IP Socket error 103: Software caused connection abort, OpenSocketCount: 6
Exception aborting multipart upload: Synapse TCP/IP Socket error 103: Software caused connection abort
Exception Operation Canceled at 0000000001394B76.
Filename: /data/SOURCE/BE/Q S01 2016-2017/160921 BE SHOWNAME CLEAN.mxf
Checkpoints: TGGetOrPutEX:Operation Canceled
Now copying to main destination: S3://name-of-the-bucket/watchfolder/_to_transcode/BE/Q S01 2016-2017/160922 BE SHOWNAME CLEAN.mxf
Uploading to 160922 BE SHOWNAME CLEAN.mxf with chunk size of 67108864 Bytes and 3 Threads. File Size:    18 GB
sslshutdown Error Message: 167772287 error:0A00007F:SSL routines::bad write retry
Line 1191 in ../ssl/record/rec_layer_s3.c: ssl3_write_pending; , 1
sslshutdown Error Message: 167772287 error:0A00007F:SSL routines::bad write retry
Line 1191 in ../ssl/record/rec_layer_s3.c: ssl3_write_pending; , 1
sslshutdown Error Message: 167772287 error:0A00007F:SSL routines::bad write retry
Line 1191 in ../ssl/record/rec_layer_s3.c: ssl3_write_pending; , 1
Exception with multipart upload: Operation Canceled, CP: S3PutMPEX:Operation Canceled
Now deleting the incomplete upload: watchfolder/_to_transcode/BE/Q%20S01%202016-2017/160922%20BE%20SHOWNAME%20CLEAN.mxf?uploadId=eZoWvj1nu7Ij071R19SE4CuCR.FuMRBYP83VJDZTtGMU4gyHLYQeb0O5plG3gbytoFGYdrAuoU2UrNZ9siwfrbKDMz9PNOqVRWlL3i2v1zzcYKVyROc8UGEclB9wXO.7
10/07/2025 14:31:53 raising Synapse exception Synapse TCP/IP Socket error 103: Software caused connection abort, OpenSocketCount: 5
Exception aborting multipart upload: Synapse TCP/IP Socket error 103: Software caused connection abort
Exception Operation Canceled at 0000000001394B76.
Filename: /data/SOURCE/BE/Q S01 2016-2017/160922 BE SHOWNAME CLEAN.mxf
Checkpoints: TGGetOrPutEX:Operation Canceled

Errors / Access problems with 1 File:

/BE/Q S01 2016-2017/160926 BE CLEAN.mxf (File or drive/device/network no longer exists)

Main Copying Phase Ended At 10/07/2025 14:31:53 And Took 00:01:17


Operation Canceled - webUIcancel1

SUMMARY
------------------------------------------------------------------
Short Results: INCOMPLETE: 2 copied (31.9GB) of 113, Errors with 4 File(s), Duration: 00:01:56
Operation completed at 14:31:53 on 10/07/2025
Total duration: 00:01:56

Copied To Right Side: 2 (31.9GB)
Errors with 4 File(s).

Overall average transfer rate  : 423.1 MB/s
Average copying time per file  : 33.8 s
Copying time for fastest file  : 8.1 s
Copying time for slowest file  : 59.4 s

Average transfer rate per file : 495596.9 kB/sec
Transfer rate for slowest file : 315076.7 kB/sec
Transfer rate for fastest file : 1817328.3 kB/sec
For the fastest/slowest rates, only files at least 1 kilobyte in size were counted.


Local time zone offset :         2.0 hours.
Internet protocol timeout :      60 seconds
Max. Internet protocol retries : 2
SQLite version :                 3.49.1, thread safety level: 1
Time used for free space checks on left side: 0.0001 seconds for 2 space checks.
Time used for free space checks on right side: 0.0001 seconds for 2 space checks.

------------------------------------------------------------------

Profile Settings From /etc/.Syncovery/Syncovery.cfg

[General]
Name=TEST
LastModified=10/07/2025 14:29:51
Disabled=Yes
LeftPath=/data/TEST
RightPath=S3://name-of-the-bucket/watchfolder/_to_transcode
UseAdditionalDestinations=Yes
AdditionalDestinations={"mode":"3","atmbeh":"1","flags":"0","destinations":[{"path":"/data/DONE","flags":"1"}]}
Subfolders=Yes
LeftToRight=Yes
RightToLeft=No
CaseSensitive=Yes
UseSubfolderSelection=No
MoveFilesMode=mmOverwriteDest
MoveByCopying=Yes

[Comparison]
AdjustFolderTimes=No
DetectHardLinks=Yes
SizeConflictAction=scaCopy

[Files]
DetectMovedFiles=No
AllowMovesBetweenUsers=Yes

[Folders]
ScanAllDestinationFoldersToFindMovedFiles=No

[Internet]
RightFTPSettings=S3:Flags=UTF8+DetectTimezone+AvoidIPV6+EmulFold,"Proxy=no",Public=N,TLS=Y,StorageClass=0,RecursLst=Y
RightUser=AKIAZ3QC7UZXWAVYAHX6

[Inclusion Masks]
FileMasks=*.mxf;*.MXF;*.mov;*.MOV

[Exclusion Masks]
ExcludeMasksIgnoreTotally=Yes

[General Filters]
HiddenFiles=No
HiddenFolders=No

[Safety->Attended]
WarnLarger=Yes

[Safety->Unattended]
UnattendedDeleteMaxPercent=20

Default System Codepage: 65001
Unicode Normalization Test: Good
ACL Library Available  : Yes
Top
tobias
Posts: 1975
Joined: Tue Mar 31, 2020 7:37 pm

Re: Problems with dual destinations

Post by tobias »

Hello,
I will look into this as a possible bug, but you don't have to wait for that!

For this use case, it's better not to use the Additional Destination feature.

Instead, the local archive folder is specified as a "folder for deleted files" for the left-hand side.

Just go to the tab sheet "Files" -> "Deletions" and choose to move deleted files into a specified folder.

Specify the local archive location as a folder for the left side (L:).

That's it. Instead of deleting from the left side, the "Move Files" mode will move them to the archive folder.

We have also released Syncovery 11 a few days ago, with a much improved user interface and feature additions. You might be interested in checking it out!

Thanks.
Top
Jean-Michel
Posts: 37
Joined: Fri Jun 10, 2022 2:12 pm

Re: Problems with dual destinations

Post by Jean-Michel »

Thanks Tobias

Always great options that I forgot about as usual

Note that the Linux/Debian-Ubuntu. deb package. is still 10.18.4
Top
Post Reply

Return to “Syncovery on Linux, FreeBSD and NAS Devices”