Reflection on the management of attached files

[montuy337513]

I'm busy updating a forum that was created in 2005.
I have over 40,000 attachments. Most are photos.
I have a problem when I want to upload all the photos via FTP. FTP servers often limit to 5000 managed files.
So a feature that should be added in a future version:
Currently all files are saved in /uploads/newbb.
Consideration should be given in the future to storing files in multiple directories.
Example: /uploads/newwbb/2022/06/03/xxxxx.jpg
/uploads/newwbb/2022/06/04/xxxxx.jpg
/uploads/newwbb/2022/06/05/xxxxx.jpg
...
or any methods limiting the number of files in a directory.

This reflection will also be interesting for other modules like wggalery

[mambax7]

Can you zip the files locally, upload it, and then unzip on the server? Would it work?

[montuy337513]

Mamba, yes I can use this solution. I have the necessary knowledge and servers.
But this is not the subject of this reflection, there are physical limits set by the FAT32, NTFS, exFAT standards, ... Not to mention the limits of standards such as FTP which limits the display to 5000 files per directory.
This last point can cause backup problems for example: if you do not know these limitations, you think you have backed up your directory with the 5000 attached files. But in my client's case, he was missing more than 400,000 files. And his host does not make it easy for him.

Yes, we are all experienced developers here. We know how to do it. But this is not the case for all "our end customers".

When you buy a car, what is the first thing you do? For me, I get in the car and drive.
If I have to start adjusting the carburettor, changing the tires, putting the seats on, ... I switch to another brand.

Xoops must be able to be used in all cases, even for large sites.

[mambax7]

All valid points.
Personally I don't have any experience with creating such option, so I couldn't do it anytime soon.
If you have any ideas/suggestions on how to make it happen, maybe we could work on this together?