PCloud: Invalid files

English Support for Cloud Storage
mareike1988
Posts: 32
Joined: Thu Oct 20, 2022 1:43 pm

PCloud: Invalid files

Post by mareike1988 »

Hello,

following situation: Uploaded of files to PCloud with Syncovery version 9 fully updated under Windows.
Mode: Exact Mirror
Changed Settings:
- Date in Filename
- Verify copied files (!!!!), but not "Recopy once if verification files"
- Compression as single Zip & Encryption "zip Compatible AES"

Then:
- "Test Backup Integrity" was performed with a full check of all files.
- Several hundred files were invalid and indeed, could not be unzipped manually.


How is this possible?
- How does Syncovery ensure a correct file transfer?
- Why did "Verify copied files" not catch this?
- What should I do to ensure correct file uploads?

Thanks.

tobias
Posts: 1603
Joined: Tue Mar 31, 2020 7:37 pm

Re: PCloud: Invalid files

Post by tobias »

Hello,
thanks for the report, I will look into this and provide a detailed reply by tomorrow.

tobias
Posts: 1603
Joined: Tue Mar 31, 2020 7:37 pm

Re: PCloud: Invalid files

Post by tobias »

Hello,
I can confirm some errors, but they are bogus errors. Syncovery fails when trying to restore the Alternate Data Streams from the zip file.

The zip file itself is fine. In fact, when doing a real restore, the files are restored fine despite Syncovery logging errors and showing errors in the result summary.

Do you see lines like these in your log file too?

Error creating file E:\Tests\Docs.RestoreFromPCloud\Test.pdf:Zone.Identifier:$DATA:
Cannot create file "\\?\E:\Tests\Docs.RestoreFromPCloud\Test.pdf:Zone.Identifier:$DATA".
The filename, directory name, or volume label syntax is incorrect.

Unzipping error for Test.pdf:Zone.Identifier:$DATA, native code:19, error code:9, msg:ZipForge1:
Cannot create file "E:\Tests\Docs.RestoreFromPCloud\Test.pdf:Zone.Identifier:$DATA" - Native: 00019 - WinErr: 123

tobias
Posts: 1603
Joined: Tue Mar 31, 2020 7:37 pm

Re: PCloud: Invalid files

Post by tobias »

OK, and then there was in fact one corrupt zip file in my test.

But the upload error was reported when uploading. Did your upload log file show any errors in the result summary?

The error in my case was:
Exception uploading Tobias_2020~1.mgb.s48448891.zip:
Synapse TCP/IP Socket error 10060:
Connection timed out, in SendBuffer, FNonBlockSendTimeout=80000 (ESynapseError), CP:SynHttpsPutRaise4

and also
Synapse TCP/IP Socket error 10091

Such errors are Internet communication or server problems. Normally Syncovery deletes the incompletely uploaded file, but if the server does not accept any API requests for some time, the deletion may fail and leave an incomplete zip file on the server.

So, what we need is a complete verification in an additional profile run, similar to "Test Backup Integrity", which eventually re-uploads any corrupt files. I will check what's the best way to do this.

Question is if you had any of these really corrupted files, or just the Alternate Data Stream extraction problem mentioned in my previous reply.

tobias
Posts: 1603
Joined: Tue Mar 31, 2020 7:37 pm

Re: PCloud: Invalid files

Post by tobias »

Hello,
again the main question is: was your original upload job really error-free?

What was the result summary? For example, I had this due to connectivity or server issues:
INCOMPLETE: 222 copied (325,3MB -> 307,8MB) of 225, Errors with 3 File(s)

One problem with PCloud seems to be that incompletely uploaded files are accepted and appear on the server. As opposed to other clouds, where incomplete uploads are discarded.

mareike1988
Posts: 32
Joined: Thu Oct 20, 2022 1:43 pm

Re: PCloud: Invalid files

Post by mareike1988 »

tobias wrote:
Fri Oct 21, 2022 7:11 pm
Hello,
again the main question is: was your original upload job really error-free?

What was the result summary?
I have to check. You have to see that a full backup of my computer took weeks and many runs so many log files....

I did not check those log files. I assumed that Syncovery would fix any error on it own over time.

mareike1988
Posts: 32
Joined: Thu Oct 20, 2022 1:43 pm

Re: PCloud: Invalid files

Post by mareike1988 »

tobias wrote:
Fri Oct 21, 2022 6:53 pm
OK, and then there was in fact one corrupt zip file in my test.

But the upload error was reported when uploading. Did your upload log file show any errors in the result summary?

The error in my case was:
Exception uploading Tobias_2020~1.mgb.s48448891.zip:
Synapse TCP/IP Socket error 10060:
Connection timed out, in SendBuffer, FNonBlockSendTimeout=80000 (ESynapseError), CP:SynHttpsPutRaise4

and also
Synapse TCP/IP Socket error 10091

Such errors are Internet communication or server problems. Normally Syncovery deletes the incompletely uploaded file, but if the server does not accept any API requests for some time, the deletion may fail and leave an incomplete zip file on the server.

So, what we need is a complete verification in an additional profile run, similar to "Test Backup Integrity", which eventually re-uploads any corrupt files. I will check what's the best way to do this.

Question is if you had any of these really corrupted files, or just the Alternate Data Stream extraction problem mentioned in my previous reply.

The file was corrupted. I downloaded the ZIP from PCloud website and could not extract :-(

tobias
Posts: 1603
Joined: Tue Mar 31, 2020 7:37 pm

Re: PCloud: Invalid files

Post by tobias »

Hello,
OK thanks. I assume that these were files where the upload was interrupted, and retries failed too. I saw this in my test.

Syncovery assumes that interrupted uploads are discarded by the server, like all cloud servers do.

But PCloud obviously puts incomplete, interrupted uploads on the server like normal files. That's a big problem.

I will provide a Syncovery update in the next 1-3 days to handle the issue. You will be able to let it verify each file and re-upload only the corrupt files. It will also remember which files it has verified, so additional runs won't re-verify all the files every time.

The update will also do its best to delete incomplete files from the server. Maybe I need to make it use temporary filenames which are only renamed to the final filename when the upload is really complete (and verified if you choose the verification option).

I will post news on the issue here very soon.

You won't have to re-upload everything.

mareike1988
Posts: 32
Joined: Thu Oct 20, 2022 1:43 pm

Re: PCloud: Invalid files

Post by mareike1988 »

Thanks that gives me hope.

Did you find out why the „verify copied files“ did not catch the error? Or maybe it did but did not delete? Or maybe it never got that far?

tobias
Posts: 1603
Joined: Tue Mar 31, 2020 7:37 pm

Re: PCloud: Invalid files

Post by tobias »

Hi,
yes it never got as far as verification. It just failed on the file. And then subsequent runs didn't recognize that it was a failed upload, because PCloud just put the partial upload there as a normal file.

Post Reply