[solved] Syncovery is transfering video file before it is complete

[Jean-Michel]

Hello

I am testing Syncovery for pushing a video file exported from Adobe Premiere to an S3 bucket
As the export is a long process (multiple minutes) and asynchronous from the Syncovery scheduler (1 minute), Syncovery transfer a partial file as soon as it detects the file and close the destination object without waiting the file to be complete.

I've searched but not found if it is possible to configure Syncovery for one of those 2 behavior :

1) (preferred) : Start transferring the video file as soon as it is detected, but do not close the object while the file is still growing. This would reduce the delay for the video to be available on the S3 bucket

2) Wait for the video file encoding to be completed before performing the transfer. This is not optimal on very large file as the beginning of the transfert will be delayed until the video file is complete

Understand that I have no control on how the video file is opened by Premiere
Obviously it is not using exclusive access which could allow the 1st case to work if we could configure for example a timeout for checking that the file is not growing anymore.

Thanks

Jean-Michel

[tobias]

Hello,
Syncovery cannot start the file while it is still being written to, because the upload logic needs to know the final file size.

Therefore, please choose the checkmark "Skip Files Whose Size is Changing" in the profile under Files->More (scroll this tab sheet down to see this last, recently added checkbox).

[Jean-Michel]

It's working nicely
Thanks a lot

[ForestQ]

Could you please enable this feature by default when creating a new profile?
I synchronize with the SFTP server every minute. It took one and a half minutes to upload a large file to the server. Syncovery began synchronizing a file that had not yet finished downloading. Consequently, on my server, the partially downloaded file had a more recent timestamp and was uploaded incompletely to the SFTP server.

[tobias]

Hello,
thanks, I will think about it!

If Syncovery copies or uploads local files, it is also recommended to choose "Database-safe copying: enforce exclusive access" on the tab sheet "Access & Retries", and also turn off Volume Shadowing there.

But this only helps if the files that Syncovery needs to wait for are local. It doesn't make any difference if the files are on an SFTP server. In that case, only the option to "skip files whose size is changing" helps. Although you could also add a date/time filter such as: File Age: over 5 minutes old.

I also recommend configuring Syncovery and other software to use temporary file names while a file is being created or transferred, and then renaming the temporary file to the final filename when it's done. That way, Syncovery can be configured to ignore the temporary filenames. For example, you can add an Exclusion Mask like: *.$*

Within Syncovery, using temporary filenames is enabled on the "Files" tab sheet by choosing:
- Automatically resume (copy with temporary filename)

The temporary filename extension that Syncovery uses is .$syncovery.

Syncovery itself will always ignore any files with this extension. This will be useful if there are multiple Syncovery jobs, possibly on different machines, which work with the same folders.