Tips for managing a large number of files?
There are some very good questions here on SO about file management and storing within a large project.
Storing Images in DB - Yea or Nay?
Would you store binary data in database or in file system?
The first one having some great insights and in my project i've decided to go the file route and not the DB route.
A major point against using the filesystem is backup. But in our system we have a great backup scheme so i am not worried about that.
The next path is how to store the actual files. And I've thought about having the files' location static at all times and create a virtual directory system in the database side of things. So links to the file don't change.
The system i am building will have one global file management so all files are accessible to all users. But many that have gone the file route talk about physical directory size (if all the files are within one directory for example)
So my question is, what are some tips or best practice methods in creating folders for these static files, or if i shouldn't go the virtual directory route at all.
(the project is on the LAMP stack (PHP) if that helps at all)
Answer
One way is to assign a unique number to each file and use it to look up the actual file location. Then you an use that number to distribute files in different directories in the filesystem. For example you could use something like this scheme:
/images/{0}/{1}/{2}
{0}: file_number % 100
{1}: (file_number / 100) % 100
{2}: file_number