Syncovery Unable To Replace Characters In Folder Name

[Clyde91]

Hello All,
I am trying to use Syncovery to copy a series of folders and files from Box.com to a file share. Syncovery appears to have no issues renaming a file with invalid characters in it, but does have an issue renaming a folder with invalid characters within it. Is this not a feature in Syncovery? If not can it be added?

Below are the snippets from my job configuration file and the error presented in the log file, as I am unable to attach a text file. My specs are as follows:

Server: Server 2016
Syncovery version: 8.68g (64 bit)
Transfer Protocol: FTPS

Regards,
Virgil


Job Profile

Code: Select all

Profile Name      : FTPS_Box.com - 2020 - West

Left Base Folder  : ftp://ftp.box.com/Western Division Projects - Complete/2020 Western Division Completed Projects
Right Base Folder : \\Servername.org\ECM\BoxSync\NET_Test\2020\West

Copying Direction : Left To Right


--------------------------------
Profile Settings
--------------------------------

[General]
Name=FTPS_Box.com - 2020 - West
LastModified=8/25/2020 12:20:43 PM
Disabled=Yes
LeftPath=ftp://ftp.box.com/Western Division Projects - Complete/2020 Western Division Completed Projects
RightPath=\\Servername.org\ECM\BoxSync\NET_Test\2020\West
LeftToRight=Yes

[Comparison]
BinaryCompMode=bcmNone

[Files->More]
AutoResume=Yes

[Internet]
LeftFTPSettings=FTP3:Port=990,Passive=Y,ASCII=N,CanRenameFolders=N,AbsolutePath=Y,ForceTimestamps=N,FTPCommand=0,Flags=UTF8+ReplaceChars+UTC+DetectTimezone+ReplaceAutomatic,"Proxy=no",TLS=impl-TLS/cmd:SSL,TimZoneOfs=420/0,"Cert=none"

[Job]
Connection2=\\Servername.org\ECM\BoxSync\NET_Test\2020\West\@DOMAIN\ServiceAccount:/NN


--------------------------------
Profile XML
--------------------------------

<?xml version="1.0" encoding="UTF-8" ?>
<Profiles>
   <Profile Name="FTPS_Box.com - 2020 - West">
      <General>
         <LastModified>8/25/2020 12:20:43 PM</LastModified>
         <Disabled>Yes</Disabled>
         <LeftPath>ftp://ftp.box.com/Western Division Projects - Complete/2020 Western Division Completed Projects</LeftPath>
         <RightPath>\\Servername.org\ECM\BoxSync\NET_Test\2020\West</RightPath>
         <LeftToRight>Yes</LeftToRight>
      </General>
      <Comparison>
         <BinaryCompMode>bcmNone</BinaryCompMode>
      </Comparison>
      <Files_More>
         <AutoResume>Yes</AutoResume>
      </Files_More>
      <Internet>
         <LeftFTPSettings><Protocol>FTP3</Protocol><Port>990</Port><Passive>Y</Passive><ASCII>N</ASCII><CanRenameFolders>N</CanRenameFolders><AbsolutePath>Y</AbsolutePath><ForceTimestamps>N</ForceTimestamps><FTPCommand>0</FTPCommand><Flags>UTF8+ReplaceChars+UTC+DetectTimezone+ReplaceAutomatic</Flags><Proxy>no</Proxy><TLS>impl-TLS/cmd:SSL</TLS><TimZoneOfs>420/0</TimZoneOfs><Cert>none</Cert></LeftFTPSettings>
      </Internet>
      <Job>
         <Connection2>\\Servername.org\ECM\BoxSync\NET_Test\2020\West\@DOMAIN\ServiceAccount:/NN</Connection2>
      </Job>
   </Profile>
</Profiles>


--------------------------------
Profile Command Line
--------------------------------

Syncovery.exe ADD /Name="FTPS_Box.com - 2020 - West" /Disabled /Left="ftp://ftp.box.com/Western Division Projects - Complete/2020 Western Division Completed Projects" /Right=\\Servername.org\ECM\BoxSync\NET_Test\2020\West /L2R /BinComp=bcmNone /AutoResume /LeftFTPSettings="FTP3:Port=990,Passive=Y,ASCII=N,CanRenameFolders=N,AbsolutePath=Y,ForceTimestamps=N,FTPCommand=0,Flags=UTF8+ReplaceChars+UTC+DetectTimezone+ReplaceAutomatic,""Proxy=no"",TLS=impl-TLS/cmd:SSL,TimZoneOfs=420/0,""Cert=none""" /Connection2=\\Servername.org\ECM\BoxSync\NET_Test\2020\West\@DOMAIN\ServiceAccount:/NN 

Alternatives to ADD
   RUN (creates & runs a job without creating a permanent profile)
   DELETE "Profile Name"   (deletes a profile)
   CHANGE "Profile Name"   (changes an existing profile, keeping any existing settings which are not set on the command line)
   RENAME "Old Profile Name" "New Profile Name"

   ADD can be used to completely replace an existing profile with new settings.

   For additional parameters, such as /M (minimized) and /U (unattended), see https://www.syncovery.com/command-line/

Log File:

Code: Select all

Cannot create \\Servername.org\ECM\BoxSync\NET_Test\2020\West\Christensen 6" Line Leak Repair OPS-WO#63023018\Welding (Error 123 creating \\Servername.org\ECM\BoxSync\NET_Test\2020\West\Christensen 6" Line Leak Repair OPS-WO#63023018)
Cannot create \\Servername.org\ECM\BoxSync\NET_Test\2020\West\Christensen 6" Line Leak Repair OPS-WO#63023018\Welding\PS8088 - Welder Qualification (Forms PS7341 & PS8088) (Error 123 creating \\Servername.org\ECM\BoxSync\NET_Test\2020\West\Christensen 6" Line Leak Repair OPS-WO#63023018\Welding)
Unable to create folder \\Servername.org\ECM\BoxSync\NET_Test\2020\West\Christensen 6" Line Leak Repair OPS-WO#63023018\Welding\PS8088 - Welder Qualification (Forms PS7341 & PS8088)\, error code 123: The filename, directory name, or volume label syntax is incorrect.
8/25/2020 10:42:07 AM IP Exception (5): EFCreateError Cannot create file "\\Servername.org\ECM\BoxSync\NET_Test\2020\West\Christensen 6" Line Leak Repair OPS-WO#63023018\Welding\PS8088 - Welder Qualification (Forms PS7341 & PS8088)\Josh pitchford exp 9-23-2020.pdf.d20200511-u141700.s3564752.$syncovery". The filename, directory name, or volume label syntax is incorrect @ 0000000000524823, CP:CorEX:Cannot create file "\\Servername.org\ECM\BoxSync\NET_Test\2020\West\Christensen 6" Line Leak Repair OPS-WO#63023018\Welding\PS8088 - Welder Qualification (Forms PS7341 & PS8088)\Josh pitchford exp 9-23-2020.pdf.d20200511-u141700.s3564
8/25/2020 10:42:08 AM IP Exception (5): EFCreateError Cannot create file "\\Servername.org\ECM\BoxSync\NET_Test\2020\West\Christensen 6" Line Leak Repair OPS-WO#63023018\Welding\PS8088 - Welder Qualification (Forms PS7341 & PS8088)\Josh pitchford exp 9-23-2020.pdf.d20200511-u141700.s3564752.$syncovery". The filename, directory name, or volume label syntax is incorrect @ 0000000000524823, CP:CorEX:Cannot create file "\\Servername.org\ECM\BoxSync\NET_Test\2020\West\Christensen 6" Line Leak Repair OPS-WO#63023018\Welding\PS8088 - Welder Qualification (Forms PS7341 & PS8088)\Josh pitchford exp 9-23-2020.pdf.d20200511-u141700.s3564
8/25/2020 10:42:11 AM IP Exception (5): EFCreateError Cannot create file "\\Servername.org\ECM\BoxSync\NET_Test\2020\West\Christensen 6" Line Leak Repair OPS-WO#63023018\Welding\PS8088 - Welder Qualification (Forms PS7341 & PS8088)\Josh pitchford exp 9-23-2020.pdf.d20200511-u141700.s3564752.$syncovery". The filename, directory name, or volume label syntax is incorrect @ 0000000000524823, CP:CorEX:Cannot create file "\\Servername.org\ECM\BoxSync\NET_Test\2020\West\Christensen 6" Line Leak Repair OPS-WO#63023018\Welding\PS8088 - Welder Qualification (Forms PS7341 & PS8088)\Josh pitchford exp 9-23-2020.pdf.d20200511-u141700.s3564

[tobias]

Hello,
automatic replacement of disallowed characters has been implemented for the Box protocol. Not for FTP. I do not believe you should be using FTP with Box.

Please try changing the FTP protocol to "Box". Click the Browse button to authorize Syncovery with Box and choose the folder.

When using FTP, you need to configure the replacements manually for each character. This is done via "Replace Characters..." on the second tab sheet of the Internet/FTP dialog. I think even when using the native Box API, you should choose the automatic replacements there.

For FTP replacements, you need to specify characters or strings that do not otherwise exist, for example you could replace a double quote with $$QT$$ or a rare Unicode character that you could pick with the character map.

[Clyde91]

Hi Tobias,
I hate to bring this back up, but after setting up Syncovery with the box protocol, I initiated file copy from box to fileshare and found that Syncovery is not changing invalid characters to their ASCII counterparts. I read the logs and found the following:

Code: Select all

Created folder: \\Server.org\ECM\BoxSync\2017 Western Division Projects - Complete\1-AD-21 Replacement AFE64355 20″ WO 620005279 & 4″ WO 620005278\Example Jurisdictional Project\General\Daily Progress Reports

I verified this by checking my fileshare and found that the directories were in fact created.

Job settings were as follows:

Code: Select all

Profile Settings From C:\ProgramData\Syncovery\Syncovery.ini

[General]
Name=API_Box.com - 2017 West
LastModified=9/2/2020 12:22:52 PM
LeftPath=ext://Box/Western Division Projects - Complete/2017 Western Division Completed Projects
RightPath=\\Server.org\ECM\BoxSync\2017 Western Division Projects - Complete
LeftToRight=Yes

[Internet]
LeftFTPSettings=Ext:ExtProt=Box,Flags=UTF8+ReplaceChars+UTC+DetectTimezone+ReplaceAutomatic,"Proxy=no",Lib=0
LeftExtendedProtocolName=Box

[Job]
Connection2=\\Server.org\ECM\BoxSync\2017 Western Division Projects - Complete@Domain\Service-Account:/NN

[Schedule]
ScheduledNormally=Yes
RepeatAfter=Yes
OrigTimeOfDay=6:30:00 PM


------------------------------------------------------------------


System Info: Microsoft Windows Server 2016 Standard Build 14393
Codepage/Country/Locale: 1252/1/0409
Windows Timezone Offset: -300
OS Architecture: 64-bit
OS Language: 1033
System Device: \Device\HarddiskVolume2
System Directory: C:\Windows\system32
Windows Directory: C:\Windows

I now have some duplicate directories and some inaccessible directories. Apparently if a directory ends with a period, it is inaccessible. Let me know what must be done to fix this issue.

Regards,
Virgil

[tobias]

Hello,
if the folders could be created, that means that the characters were replaced successfully. Otherwise they cannot be created.

Some of the replacement characters may look very similar to the real ones, such as the double quotes. But they are different ones.

I agree that Windows Explorer has a problem with folders ending with a dot. Other software may or may not have problems with that. If you just want a local backup copy of the Box folders, you can leave the folders as they are. If you want to work on the files locally, I agree that the dots may need to be renamed. The only way to renamed with Syncovery would be with a PascalScript such as this:

function OnReplaceFilenameLeftToRight(const FileName: UnicodeString;
const isFolder: Boolean):UnicodeString;
begin
if (FileName>'') and (FileName[Length(FileName)]='.') then
Result:=Copy(FileName,1,Length(FileName)-1)+'__DOT__'
else
Result:=FileName;
end;

I don't know what you mean with "Duplicate Folders". Duplicate folders cannot exist on Windows.

[Clyde91]

Hello!
Thanks for the script Tobias, that did solve my folders ending with a period issue. I mis-spoke earlier about having duplicate folders. Our earlier software translated those illegal characters to percent encoding. So now I have folders and files where illegal characters have been replaced with 2 different methods (percent encoding and similar characters). We have already committed to using the percent encoding (our current project is over 800gb of files), so I have tried the "Use custom replacements" table in the Replace Characters dialog, but it does not work. I have also noticed that my characters will get mixed up with every subsequent opening of the dialog. Is there some script I can use to set the replacement characters for example from: " to %22 and so on?

I appreciate your assistance,
Virgil

[tobias]

Hello,
apparently there is a bug when trying to configure a replacement for the double quotes. I will fix that in the next update.

The replacements should be like this, do you agree?

PercentageReplacements!.png (63.55 KiB) Viewed 6644 times

You could try these even now if you leave away the double quotes.

Concerning duplicate items, if you do an Exact Mirror from Box to local, Syncovery will delete the items which are not correctly encoded.

[Clyde91]

Hi Tobias,
Thank you for looking into this for me. Unfortunately I am unable to leave out the " as that is our single most encountered illegal character. I look forward to the update.

Regards,
Virgil

[tobias]

Hello,
I fixed it and will release the update tomorrow.

[tobias]

Hello,
the new version 9.13 is now available for Windows, and it can handle the replacements from my screenshot correctly.

[Clyde91]

Hi Tobias,
Can we expect to see this fix in version 8?

Regards,
Virgil