What is the limit on malloc parameter of type size_t in C? Docs say it has an upper limit of UINT_MAX but I can't go beyond INT_MAX

Question

I want to allocate a 2.9GB char array with

  database = (char*) malloc((2900 * 1000000 * sizeof(char)));

This gives an integer overflow warning and the malloc returns NULL. The malloc parameter is of type size_t which according to documentation is of type unsigned int.

So the max should be UINT_MAX which is at least 2.9GB. However, if I try to allocate more than MAX_INT the malloc fails. Does this mean size_t on my system is of type int? How do I check this? I looked through

/usr/include/stdlib.h 

and

./lib/gcc/x86_64-redhat-linux/4.1.1/include/stddef.h 

but can't find the definition of size_t. Thanks very much

Answer 1

The parameter is of type size_t and malloc is required to accept any possible value of type size_t. Note that "accept" does not meant it is required to allocate that much; all it means is that malloc is not allowed to misinterpret a very large number you give it as a small/negative number due to overflow issues, thereby returning a buffer that's too small and creating a critical undetectable vulnerability your program cannot defend against. There are many possible reasons malloc could fail to allocate very large objects:

• that much memory is not available from the system
• due to fragmentation, no contiguous range of virtual addresses that large is available
• arbitrary limits

In this case I suspect you might be seeing the third, arbitrary limits, though I would not consider them so arbitrary. There's a very good reason to disallow allocations (and the existence of any objects) larger than SIZE_MAX/2: taking the difference between pointers within such large objects will result in (extremely dangerous) integer overflow and undefined behavior when the result does not fit in the (signed) type ptrdiff_t. Thus, on a robust 32-bit system, while the virtual address space size is 4GB, the maximum size of any single object will be 2GB.