How to split large folders into subfolders containing a certain amount of files?

[Connor0308]

Dear,

I am currently looking to get something (from my perspective) easy done. However, I am really stuck - and would be more than happy for some guidance.
On my file system, I have a folder containing roughly 150k files. What I would like to do is to have this "bunch" split into folders of 10 - 15k files each.
So further sorting required. My "pain" is simply to have import folders for a different program which simply resists to import the complete "big" folder.

What I have already tested was the "copy only XXX files" - but this did not work (it gave me the whole folder in the preview).

Thank you for pointing me into the right direction.

Best regards,

Connor

[tobias]

Hello,
I think the best way is to use a small PascalScript, which can be added to the Syncovery profile. I can write that for you. As I understand, the original folder should stay as it is?

Do the separate folders need to be updated in the future, or is this a one-time task?

Are existing files ever updated, or only new ones added?

[Connor0308]

Dear,

thank you for your swift feedback - and the willingness to support me here.
This is a one time task, and the files in the folders are never updated.
Whether the original folder is modified or not is not of relevance, since it is only a working folder.

I hope that this answered your questions.

Best regards,

Connor

[tobias]

Hello,
OK, please create a simple profile (Standard Copying) that copies from left to right, into an empty folder on the right side.

Change the number of files to copy in parallel to 1 on the Files tab sheet. This will ensure that the script counts correctly.

Then add the following script in the profile via Job->PascalScript. That's it. You can run the profile and it will sort the files into subfolders named 1,2,3,4,5 etc. with 10,000 files in each.

Code: Select all

const MaxPerFolder=10000;

var FolderNum, Counter:Integer;

function OnBeforeFileCopy(const DirectionIsL2R:Boolean;
        var Source,Dest,DestPath,LeftSubPath,RightSubPath:UnicodeString):Boolean;
var AddFolderName,NewDestPath,NewDest,NewRightSubPath:UnicodeString;
begin
  Result:=true;
  if not DirectionIsL2R then
    Exit;
  
  if Counter mod MaxPerFolder=0 then 
     Inc(FolderNum);
  
  Inc(Counter);

  AddFolderName:=IntToStr(FolderNum);
  NewDestPath:=ConcatPathWithDelim(DestPath,AddFolderName,PathDelim);
  NewDest:=ConcatPathWithDelim(NewDestPath,ExtractFileName(Dest),PathDelim);
  NewRightSubPath:=IncludeLeadingPathDelim(ConcatPathWithDelim(RightSubPath,AddFolderName,PathDelim));
 
  if not MakeSurePathExists(NewDestPath,true) then begin
    Log('Could not create '+NewDestPath);
    Exit;
    end;
 
  Dest:=NewDest;
  DestPath:=NewDestPath;
  RightSubPath:=NewRightSubPath;
  end;

[Connor0308]

Hello,

thank you for your swift support. Just entered it and it is currently running.
What I can already see is that a folder 1 was created, so I assume that it is working as intended.
And I just realise that I need to rediscover my Pascal abilities .

Once again - thank you for your great support on a Sunday.

Best regards

Connor