I am unable to connect to a working SMB share on my EMC Unity array using Syncovery. I first attempted to install the native client on a synology nas and was unable to connect to it so I spun up an Ubuntu 20.04 VM and tried that way. Same issue.
I can connect to an SMB share on my synology nas without issue. I can connect using SMBCLIENT from the cli on the ubuntu VM.
Syncovery just fails
09/19/2021 08:21:00 ERROR (EPrepException): Cannot access smb://myserver/myshare
(Samba Open Directory Error Codes 5406637/25 for smb://myserver at 0000000000F5BEF9) @ 0000000000E648FB Thread ID: 1967720192
I have no idea whats going on with it. Are there any other logs that I can look at that might show what its trying to do and what its actually failing out at?
I can connect to this SMB share using the syncovery windows client and perform a sync but windows is notoriously slow at file enumeration and i have 20 million files to sync
Cannot connect to SMB Share on EMC Unity Array
Re: Cannot connect to SMB Share on EMC Unity Array
Hello,
the internal SMB button in Syncovery for Linux is just a quick and dirty way to connect to SMB shares. It might be sufficient for some low profile use cases, but it will be much slower than the Windows version, as it can do single-threaded operations only (due to using libsmbclient).
You really need to mount the smb folder to a local mount point, and specify that local mount point in Syncovery.
But I don't know if the Linux version will be faster in folder scanning. Listing speed can be increased even on Windows, by increasing the number of scanning threads on the Performance tab sheet on the Program Settings dialog, or on the Job tab sheet in the profile.
For really fast scanning, you need to install our Syncovery Remote Service directly on the storage. Probably impossible with the Dell EMC Unity. I have no idea what such storage systems are good for if you can't get a lightning fast complete recursive listing of all folders. SMB simply doesn't support that. It's only possible with our Remote Service.
See also
https://www.syncovery.com/documentation/faq/fastlist/
the internal SMB button in Syncovery for Linux is just a quick and dirty way to connect to SMB shares. It might be sufficient for some low profile use cases, but it will be much slower than the Windows version, as it can do single-threaded operations only (due to using libsmbclient).
You really need to mount the smb folder to a local mount point, and specify that local mount point in Syncovery.
But I don't know if the Linux version will be faster in folder scanning. Listing speed can be increased even on Windows, by increasing the number of scanning threads on the Performance tab sheet on the Program Settings dialog, or on the Job tab sheet in the profile.
For really fast scanning, you need to install our Syncovery Remote Service directly on the storage. Probably impossible with the Dell EMC Unity. I have no idea what such storage systems are good for if you can't get a lightning fast complete recursive listing of all folders. SMB simply doesn't support that. It's only possible with our Remote Service.
See also
https://www.syncovery.com/documentation/faq/fastlist/
Re: Cannot connect to SMB Share on EMC Unity Array
Hello again,
of course ways to speed this up depend very much on the copying direction(s).
If you copy to the NAS only, you can use "Cache Destination File List".
If you copy from the NAS, or both ways, you can use Real-Time Sync, but on Windows only. It defaults to making one complete scan and then processing only folders with changes, all the time. This is not available on Syncovery for Linux.
Again speed up the full listings by increasing the number of threads.
Concerning real-time, please see:
https://www.syncovery.com/real-time-sync/
of course ways to speed this up depend very much on the copying direction(s).
If you copy to the NAS only, you can use "Cache Destination File List".
If you copy from the NAS, or both ways, you can use Real-Time Sync, but on Windows only. It defaults to making one complete scan and then processing only folders with changes, all the time. This is not available on Syncovery for Linux.
Again speed up the full listings by increasing the number of threads.
Concerning real-time, please see:
https://www.syncovery.com/real-time-sync/
Re: Cannot connect to SMB Share on EMC Unity Array
Yes, unfortunately I can't install anything on the EMC (they don't actually give you Root access
)
I've increased the threads that hasnt helped much as far as I can tell.
My assumption is that my speed is due to file enumeration but I'm not even sure to be honest. The issue that I'm having is that my file transfer rate is < 15MBps. This is moving across a 10gb network from ultrafast SSD to 7200RPM HDD. A single large file moves across this link at ~100mbps. These millions of small files are considerably slower. This is expected as multiple small files are always slower than single large files, but im trying to do anything I can to speed it up.
This is one way, Real Time Sync, no deletion on destination. I have set the do not scan destination and used the two archive flags as mentioned in the link you posted above.
Any other ideas?
![Sad :(](./images/smilies/icon_e_sad.gif)
I've increased the threads that hasnt helped much as far as I can tell.
My assumption is that my speed is due to file enumeration but I'm not even sure to be honest. The issue that I'm having is that my file transfer rate is < 15MBps. This is moving across a 10gb network from ultrafast SSD to 7200RPM HDD. A single large file moves across this link at ~100mbps. These millions of small files are considerably slower. This is expected as multiple small files are always slower than single large files, but im trying to do anything I can to speed it up.
This is one way, Real Time Sync, no deletion on destination. I have set the do not scan destination and used the two archive flags as mentioned in the link you posted above.
Any other ideas?
Re: Cannot connect to SMB Share on EMC Unity Array
Hello,
for many small files, you just need to increase the number of "files to copy in parallel" on the Files tab sheet. You can easily set it to 10-30 on a LAN.
Also on the Files tab sheet, make sure you did NOT choose "Bypass File Buffering". That would be very bad.
But, something that can speed up copying with some NAS devices is under Files->More:
scroll down and remove the checkmark
- Check Destination File (Again) During Copying Phase
for many small files, you just need to increase the number of "files to copy in parallel" on the Files tab sheet. You can easily set it to 10-30 on a LAN.
Also on the Files tab sheet, make sure you did NOT choose "Bypass File Buffering". That would be very bad.
But, something that can speed up copying with some NAS devices is under Files->More:
scroll down and remove the checkmark
- Check Destination File (Again) During Copying Phase
Re: Cannot connect to SMB Share on EMC Unity Array
Making these changes has made a massive difference. I am in the 50mbps range now. Thank you! I was using a trial version but now that this is sorted out, ill definitely be making the purchase.tobias wrote: ↑Sun Sep 19, 2021 3:11 pmHello,
for many small files, you just need to increase the number of "files to copy in parallel" on the Files tab sheet. You can easily set it to 10-30 on a LAN.
Also on the Files tab sheet, make sure you did NOT choose "Bypass File Buffering". That would be very bad.
But, something that can speed up copying with some NAS devices is under Files->More:
scroll down and remove the checkmark
- Check Destination File (Again) During Copying Phase