Wasabi >32MB files

English Support for Cloud Storage
Post Reply
lychee
Posts: 6
Joined: Sun Jun 06, 2021 7:20 pm

Wasabi >32MB files

Post by lychee »

Hi folks,

Wonder if anyone has any pointers for me please?

I use Syncovery on windows server to sync to S3 storage on Wasabi. This worked great until 4-5 days ago when and all of my jobs fail with errors like the section at the bottom. I don't believe anything changed in my configuration.

After some testing this only seems to happen on files larger than 32MB in size. Anything larger than that size fails, but smaller files work no problem. I am able to upload the same files that fail to the same folders using other products using the same credentials so this seems specific to Syncovery.

I upgraded to the latest version a day or so ago to see if an update resolved the problem (but it didn't), so I am on 9.36 build 215. I cant remember which version I was previously, but it had been 2-3 months since the last time I updated. I tried increasing the size in program settings / cloud / maximum part size which was set to 32MB, but that hasn't helped.

Any help would be appreciated - thanks !

---
Uploading to 911c3d92.d20210602-u033052.tar with chunk size of 33554432 Bytes and 1 Threads. File Size: 174,550 kB
06/06/2021 16:05:16 TBlockSocket.RecvPacket: no data received from WaitingData, setting error to 10054
Retrying after exception: Synapse TCP/IP Socket error 10054: Connection reset by peer, Retry Number: 0
Information: synsock.Select took 60.97 seconds (max=650).
06/06/2021 16:06:18 TBlockSocket.RecvPacket: no data received from WaitingData, setting error to 10054
Retrying after exception: Synapse TCP/IP Socket error 10054: Connection reset by peer, Retry Number: 1
06/06/2021 16:07:16 TBlockSocket.RecvPacket: no data received from WaitingData, setting error to 10054
Retrying after exception: Synapse TCP/IP Socket error 10054: Connection reset by peer, Retry Number: 2
06/06/2021 16:08:17 TBlockSocket.RecvPacket: no data received from WaitingData, setting error to 10054
Retrying after exception: Synapse TCP/IP Socket error 10054: Connection reset by peer, Retry Number: 3
Information: synsock.Select took 61.80 seconds (max=650).
06/06/2021 16:09:19 TBlockSocket.RecvPacket: no data received from WaitingData, setting error to 10054
Retrying after exception: Synapse TCP/IP Socket error 10054: Connection reset by peer, Retry Number: 4
06/06/2021 16:10:17 TBlockSocket.RecvPacket: no data received from WaitingData, setting error to 10054
Retrying after exception: Synapse TCP/IP Socket error 10054: Connection reset by peer, Retry Number: 5
06/06/2021 16:11:14 TBlockSocket.RecvPacket: no data received from WaitingData, setting error to 10054
Retrying after exception: Synapse TCP/IP Socket error 10054: Connection reset by peer, Retry Number: 6
06/06/2021 16:12:10 TBlockSocket.RecvPacket: no data received from WaitingData, setting error to 10054
Retrying after exception: Synapse TCP/IP Socket error 10054: Connection reset by peer, Retry Number: 7
06/06/2021 16:13:11 TBlockSocket.RecvPacket: no data received from WaitingData, setting error to 10054
Retrying after exception: Synapse TCP/IP Socket error 10054: Connection reset by peer, Retry Number: 8
Information: synsock.Select took 60.05 seconds (max=650).
06/06/2021 16:14:13 TBlockSocket.RecvPacket: no data received from WaitingData, setting error to 10054
Retrying after exception: Synapse TCP/IP Socket error 10054: Connection reset by peer, Retry Number: 9
06/06/2021 16:15:10 TBlockSocket.RecvPacket: no data received from WaitingData, setting error to 10054
Retrying after exception: Synapse TCP/IP Socket error 10054: Connection reset by peer, Retry Number: 10
06/06/2021 16:16:07 TBlockSocket.RecvPacket: no data received from WaitingData, setting error to 10054
Retrying after exception: Synapse TCP/IP Socket error 10054: Connection reset by peer, Retry Number: 11
06/06/2021 16:17:05 TBlockSocket.RecvPacket: no data received from WaitingData, setting error to 10054
Retrying after exception: Synapse TCP/IP Socket error 10054: Connection reset by peer, Retry Number: 12
06/06/2021 16:18:07 TBlockSocket.RecvPacket: no data received from WaitingData, setting error to 10054
Retrying after exception: Synapse TCP/IP Socket error 10054: Connection reset by peer, Retry Number: 13
06/06/2021 16:19:07 TBlockSocket.RecvPacket: no data received from WaitingData, setting error to 10054
Retrying after exception: Synapse TCP/IP Socket error 10054: Connection reset by peer, Retry Number: 14
06/06/2021 16:20:06 TBlockSocket.RecvPacket: no data received from WaitingData, setting error to 10054
Retrying after exception: Synapse TCP/IP Socket error 10054: Connection reset by peer, Retry Number: 15
06/06/2021 16:21:05 TBlockSocket.RecvPacket: no data received from WaitingData, setting error to 10054
Retrying after exception: Synapse TCP/IP Socket error 10054: Connection reset by peer, Retry Number: 16
06/06/2021 16:22:04 TBlockSocket.RecvPacket: no data received from WaitingData, setting error to 10054
Retrying after exception: Synapse TCP/IP Socket error 10054: Connection reset by peer, Retry Number: 17
Information: synsock.Select took 60.55 seconds (max=650).
06/06/2021 16:23:07 TBlockSocket.RecvPacket: no data received from WaitingData, setting error to 10054
Retrying after exception: Synapse TCP/IP Socket error 10054: Connection reset by peer, Retry Number: 18
06/06/2021 16:24:11 TBlockSocket.RecvPacket: no data received from WaitingData, setting error to 10054
Retrying after exception: Synapse TCP/IP Socket error 10054: Connection reset by peer, Retry Number: 19
06/06/2021 16:25:10 TBlockSocket.RecvPacket: no data received from WaitingData, setting error to 10054
Exception with multipart upload: Synapse TCP/IP Socket error 10054: Connection reset by peer

tobias
Posts: 1603
Joined: Tue Mar 31, 2020 7:37 pm

Re: Wasabi >32MB files

Post by tobias »

Hello,
I'm not seeing such errors here, maybe it is an intermittent problem with servers or Internet connection.

You could try reducing the part size on the Program Settings dialog, tab sheet "Cloud". For example, try 8 MB.

lychee
Posts: 6
Joined: Sun Jun 06, 2021 7:20 pm

Re: Wasabi >32MB files

Post by lychee »

Hello Tobias,

Thank you for the reply, with that setting set to:

8MB, I can upload a 7Mb file, but a 15Mb file and 34MB file fails

16MB, the same 7Mb and 15MB file works ok but 34Mb fails

64MB, - same as 32MB

I can upload the same file from the same machine with s3 browser. I also built a clean VM , installed a demo copy of Syncovery and have the same problem.

lychee
Posts: 6
Joined: Sun Jun 06, 2021 7:20 pm

Re: Wasabi >32MB files

Post by lychee »

I can also modify the same profile to another s3 bucket (storj.io) and all files upload ok, so this seems to be specific to Syncovery with Wasabi

tobias
Posts: 1603
Joined: Tue Mar 31, 2020 7:37 pm

Re: Wasabi >32MB files

Post by tobias »

Hello,
the reason is that Syncovery uses the multi-part upload API for files over 32MB, and that API seems to be broken in your Wasabi account.

I will send you an updated Syncovery version to test so you can upload files up to 5GB without multi-part. But I think you should contact Wasabi support and ask them why you get timeouts after 60 seconds when Syncovery uploads parts.

It doesn't happen with my own Wasabi account.

lychee
Posts: 6
Joined: Sun Jun 06, 2021 7:20 pm

Re: Wasabi >32MB files

Post by lychee »

Thank you Tobias, I will contact them in the morning :idea:

lychee
Posts: 6
Joined: Sun Jun 06, 2021 7:20 pm

Re: Wasabi >32MB files

Post by lychee »

Hi Tobias,

I have resolved this.

The url for my profile was set to S3://<bucket>@s3.wasabisys.com/<folder> as it has been for the past 2 years and worked ok until last week.

I have modified this to be the new (?) regional format service URL for my bucket which is in the eu-central-01 region which I found here https://wasabi-support.zendesk.com/hc/e ... t-regions- and it is working again with the maximum part size set to 32MB.

As a test I created a new profile in Syncovery to point to wasabi and it defaults to a the service URL of s3.wasabisys.com which fails to back up for me. The url can obviously be modified after the profile is created but maybe a region drop down is needed for this provider ?

Thanks again for the help

tobias
Posts: 1603
Joined: Tue Mar 31, 2020 7:37 pm

Re: Wasabi >32MB files

Post by tobias »

Hello,
thanks for the info. I wasn't actually aware that there were multiple different endpoints for Wasabi. I will update Syncovery to notify the user about this.

tobias
Posts: 1603
Joined: Tue Mar 31, 2020 7:37 pm

Re: Wasabi >32MB files

Post by tobias »

Hello,
the new version 9.36a now offers multiple predefined Wasabi regions to avoid such problems in the future.

lychee
Posts: 6
Joined: Sun Jun 06, 2021 7:20 pm

Re: Wasabi >32MB files

Post by lychee »

I downloaded it earlier today - looks good :)

Post Reply