I have a sync job that copies the files from a share to a MinIO S3 server. Of the 6TB in the share about 2,4TB are uploaded every time the job is running. So Syncovery (v11.1.4 build 41) creates a new version of the files that are being copied.
Share is a Windows Server 2022, latest version of MinIO runs on an Ubuntu 24.04 server.
It are always the same files. This is an example of a file that has been uploaded 4 times now. File is not changed in years.
v4 CURRENT VERSION
2cbfebe1-5223-40dc-a441-ea8b4c02ba9a
Last modified: Wed, Mar 12 2025 09:59:44 (GMT+1)Size: 9.0 MiB
v3
519633bb-f44e-4182-8359-d5c48a89edbc
Last modified: Tue, Mar 11 2025 10:38:28 (GMT+1)Size: 9.0 MiB
v2
26e9b5bd-9774-4efa-b268-9b548a6f22bd
Last modified: Mon, Mar 10 2025 19:13:56 (GMT+1)Size: 9.0 MiB
v1
4f712114-fb32-4132-99a2-4cfa7b5a58d5
Last modified: Mon, Mar 10 2025 06:53:32 (GMT+1)Size: 9.0 MiB
As this S3 server has immunity enabled this makes the sync useless as it would fill the server in months.
Anybody an idea what is going on?
Syncovery keeps creating new versions on MinIO S3 storage
Re: Syncovery keeps creating new versions on MinIO S3 storage
Hello,
please give me a sample file name. If the file has a name that was previously encoded by Syncovery, you need to disabled decoding of any file names under Versioning -> More.
Further tips:
Please run the job in Attended Mode and look at the Sync Preview. Scroll to the file in question (or just one sample file).
Compare the file details on the left and right sides.
Are they shown for both sides? If yes, what's the difference? If there's one, you need to make Syncovery ignore it.
Are the details missing for the destination? Then apparently the file listing isn't working correctly, and we need to look into it.
please give me a sample file name. If the file has a name that was previously encoded by Syncovery, you need to disabled decoding of any file names under Versioning -> More.
Further tips:
Please run the job in Attended Mode and look at the Sync Preview. Scroll to the file in question (or just one sample file).
Compare the file details on the left and right sides.
Are they shown for both sides? If yes, what's the difference? If there's one, you need to make Syncovery ignore it.
Are the details missing for the destination? Then apparently the file listing isn't working correctly, and we need to look into it.
Re: Syncovery keeps creating new versions on MinIO S3 storage
Filename of this particular file is 0000001.tif, but this goes also for pdf, jpg, png files.
File is not encoded by Syncovery as far as I know and can see as it appears as 0000001.tif on the Objec browser of MinIO
When run in Attended mode I see no files on the left side. So Syncovery does not see the files on the S3 storage which is odd as the MinIO Object browser (webinterface) and also a Windows S3 Browser sees all the files in the folder.
I have 3 more syncs to the S3 storage and all 3 work as expected, only this one is not working correctly. I recreated the sync but that did not solve the issue.
File is not encoded by Syncovery as far as I know and can see as it appears as 0000001.tif on the Objec browser of MinIO
When run in Attended mode I see no files on the left side. So Syncovery does not see the files on the S3 storage which is odd as the MinIO Object browser (webinterface) and also a Windows S3 Browser sees all the files in the folder.
I have 3 more syncs to the S3 storage and all 3 work as expected, only this one is not working correctly. I recreated the sync but that did not solve the issue.
Re: Syncovery keeps creating new versions on MinIO S3 storage
Please check if a Syncovery update is available.
Turn on Internet Protocol Logging and run the job, cancel preview, and send all the logs to support@syncovery.com.
Turn on Internet Protocol Logging and run the job, cancel preview, and send all the logs to support@syncovery.com.
Re: Syncovery keeps creating new versions on MinIO S3 storage
This log files include information about our customers so that is not something I would like to share.
However I checked the log files created and what I see is that the scanning stops after a folder that had been excluded had been scanned.
eg
Scan starts:
S3: Backup/<Folder 1>/Archive 0 <DIR>
S3: Backup/<Folder 1>/Archive/<other folder> 0 <DIR>
S3: Backup/<Folder 1>/Archive/<other folder>/A 0 <DIR>
S3: Backup/<Folder 1>/Archive/<other folder>/A/A0000005.png 2491 <file>
More folders and files of <Folder 1>
S3: Backup/<Folder 2>/Archive 0 <DIR>
Scanning of <Folder 2>
.
.
.
.
Gets to
S3: Backup/<Folder 11>/Archive 0 <DIR>
S3: Backup/<Folder 11>/Archive/<other folder> 0 <DIR>
S3: Backup/<Folder 11>/Archive/<other folder>/A 0 <DIR>
S3: Backup/<Folder 11>/Archive/<other folder>/A/A0000004.jpg 23491 <file>
<Folder 11> is on the Global exclusion list as Don't copy. After scanning the content of that folder the scanning of the S3 storage stopped.
I removed the folder (and a couple of others) from the Global exclusion list and added them to the profile and the profile is now running as expected.
However I checked the log files created and what I see is that the scanning stops after a folder that had been excluded had been scanned.
eg
Scan starts:
S3: Backup/<Folder 1>/Archive 0 <DIR>
S3: Backup/<Folder 1>/Archive/<other folder> 0 <DIR>
S3: Backup/<Folder 1>/Archive/<other folder>/A 0 <DIR>
S3: Backup/<Folder 1>/Archive/<other folder>/A/A0000005.png 2491 <file>
More folders and files of <Folder 1>
S3: Backup/<Folder 2>/Archive 0 <DIR>
Scanning of <Folder 2>
.
.
.
.
Gets to
S3: Backup/<Folder 11>/Archive 0 <DIR>
S3: Backup/<Folder 11>/Archive/<other folder> 0 <DIR>
S3: Backup/<Folder 11>/Archive/<other folder>/A 0 <DIR>
S3: Backup/<Folder 11>/Archive/<other folder>/A/A0000004.jpg 23491 <file>
<Folder 11> is on the Global exclusion list as Don't copy. After scanning the content of that folder the scanning of the S3 storage stopped.
I removed the folder (and a couple of others) from the Global exclusion list and added them to the profile and the profile is now running as expected.
Re: Syncovery keeps creating new versions on MinIO S3 storage
Hi,
glad you solved it. There must have been something wrong in the exclusions, so they affected only one side. For example, you cannot use forward slashes (/) in exclusions.
Without the original logs, I cannot look into this further, so I think we can consider the problem solved.
glad you solved it. There must have been something wrong in the exclusions, so they affected only one side. For example, you cannot use forward slashes (/) in exclusions.
Without the original logs, I cannot look into this further, so I think we can consider the problem solved.