Page 1 of 1
How to split large folders into subfolders containing a certain amount of files?
Posted: Sun Apr 11, 2021 12:12 pm
by Connor0308
Dear,
I am currently looking to get something (from my perspective) easy done. However, I am really stuck - and would be more than happy for some guidance.
On my file system, I have a folder containing roughly 150k files. What I would like to do is to have this "bunch" split into folders of 10 - 15k files each.
So further sorting required. My "pain" is simply to have import folders for a different program which simply resists to import the complete "big" folder.
What I have already tested was the "copy only XXX files" - but this did not work (it gave me the whole folder in the preview).
Thank you for pointing me into the right direction.
Best regards,
Connor
Re: How to split large folders into subfolders containing a certain amount of files?
Posted: Sun Apr 11, 2021 1:21 pm
by tobias
Hello,
I think the best way is to use a small PascalScript, which can be added to the Syncovery profile. I can write that for you. As I understand, the original folder should stay as it is?
Do the separate folders need to be updated in the future, or is this a one-time task?
Are existing files ever updated, or only new ones added?
Re: How to split large folders into subfolders containing a certain amount of files?
Posted: Sun Apr 11, 2021 1:26 pm
by Connor0308
Dear,
thank you for your swift feedback - and the willingness to support me here.
This is a one time task, and the files in the folders are never updated.
Whether the original folder is modified or not is not of relevance, since it is only a working folder.
I hope that this answered your questions.
Best regards,
Connor
Re: How to split large folders into subfolders containing a certain amount of files?
Posted: Sun Apr 11, 2021 2:19 pm
by tobias
Hello,
OK, please create a simple profile (Standard Copying) that copies from left to right, into an empty folder on the right side.
Change the number of files to copy in parallel to 1 on the Files tab sheet. This will ensure that the script counts correctly.
Then add the following script in the profile via Job->PascalScript. That's it. You can run the profile and it will sort the files into subfolders named 1,2,3,4,5 etc. with 10,000 files in each.
Code: Select all
const MaxPerFolder=10000;
var FolderNum, Counter:Integer;
function OnBeforeFileCopy(const DirectionIsL2R:Boolean;
var Source,Dest,DestPath,LeftSubPath,RightSubPath:UnicodeString):Boolean;
var AddFolderName,NewDestPath,NewDest,NewRightSubPath:UnicodeString;
begin
Result:=true;
if not DirectionIsL2R then
Exit;
if Counter mod MaxPerFolder=0 then
Inc(FolderNum);
Inc(Counter);
AddFolderName:=IntToStr(FolderNum);
NewDestPath:=ConcatPathWithDelim(DestPath,AddFolderName,PathDelim);
NewDest:=ConcatPathWithDelim(NewDestPath,ExtractFileName(Dest),PathDelim);
NewRightSubPath:=IncludeLeadingPathDelim(ConcatPathWithDelim(RightSubPath,AddFolderName,PathDelim));
if not MakeSurePathExists(NewDestPath,true) then begin
Log('Could not create '+NewDestPath);
Exit;
end;
Dest:=NewDest;
DestPath:=NewDestPath;
RightSubPath:=NewRightSubPath;
end;
Re: How to split large folders into subfolders containing a certain amount of files?
Posted: Sun Apr 11, 2021 3:19 pm
by Connor0308
Hello,
thank you for your swift support. Just entered it and it is currently running.
What I can already see is that a folder 1 was created, so I assume that it is working as intended.
And I just realise that I need to rediscover my Pascal abilities
.
Once again - thank you for your great support on a Sunday.
Best regards
Connor