In a SharePoint document library, is there any way to store the documents on the file system instead of the database?
You can make SharePoint store its documents outside the DB using the SharePoint External Storage API. In other words, this isn't some functionality accessible through the SharePoint interface, but the system does offer a way of building it yourself. There are third party providers like StoragePoint that will do the job for you.