How big can a memory-mapped file be?

mmap memory-mapped-files
user88185 picture user88185 · Apr 7, 2009 · Viewed 23.4k times · Source

What limits the size of a memory-mapped file? I know it can't be bigger than the largest continuous chunk of unallocated address space, and that there should be enough free disk space. But are there other limits?

Answer

Jason S picture Jason S · Apr 7, 2009

You're being too conservative: A memory-mapped file can be larger than the address space. The view of the memory-mapped file is limited by OS memory constraints, but that's only the part of the file you're looking at at one time. (And I guess technically you could map multiple views of discontinuous parts of the file at once, so aside from overhead and page length constraints, it's only the total # of bytes you're looking at that poses a limit. You could look at bytes [0 to 1024] and bytes [240 to 240 + 1024] with two separate views.)

In MS Windows, look at the MapViewOfFile function. It effectively takes a 64-bit file offset and a 32-bit length.

Related questions

When should I use mmap for file access?

POSIX environments provide at least two ways of accessing files. There's the standard system calls open(), read(), write(), and friends, but there's also the option of using mmap() to map the file into virtual memory. When is it preferable to …

Read More
mmap() vs. reading blocks

I'm working on a program that will be processing files that could potentially be 100GB or more in size. The files contain sets of variable length records. I've got a first implementation up and running and am now looking towards …

Read More
Setting up Laravel on a Mac php artisan migrate error: No such file or directory

Pulled a perfectly working laravel project from a git into a mac running MAMP. Project ran perfectly on a linux machine. composer install php artisan migrate, got the following error: [PDOException] SQLSTATE[HY000] [2002] No such file or directory NB: php …

Read More