How much memory should you be able to allocate?

c++ new-operator dynamic-memory-allocation
Bill picture Bill · Jun 23, 2009 · Viewed 11.2k times · Source

Background: I am writing a C++ program working with large amounts of geodata, and wish to load large chunks to process at a single go. I am constrained to working with an app compiled for 32 bit machines. The machine I am testing on is running a 64 bit OS (Windows 7) and has 6 gig of ram. Using MS VS 2008.

I have the following code:

byte* pTempBuffer2[3];
try
{
    //size_t nBufSize = nBandBytes*m_nBandCount;
    pTempBuffer2[0] = new byte[nBandBytes];
    pTempBuffer2[1] = new byte[nBandBytes];
    pTempBuffer2[2] = new byte[nBandBytes];
}
catch (std::bad_alloc)
{
    // If we didn't get the memory just don't buffer and we will get data one
    // piece at a time.
    return;
}

I was hoping that I would be able to allocate memory until the app reached the 4 gigabyte limit of 32 bit addressing. However, when nBandBytes is 466,560,000 the new throws std::bad_alloc on the second try. At this stage, the working set (memory) value for the process is 665,232 K So, it I don't seem to be able to get even a gig of memory allocated.

There has been some mention of a 2 gig limit for applications in 32 bit Windows which may be extended to 3 gig with the /3GB switch for win32. This is good advice under that environment, but not relevant to this case.

How much memory should you be able to allocate under the 64 bit OS with a 32 bit application?

Answer

jalf picture jalf · Jun 23, 2009

As much as the OS wants to give you. By default, Windows lets a 32-bit process have 2GB of address space. And this is split into several chunks. One area is set aside for the stack, others for each executable and dll that is loaded. Whatever is left can be dynamically allocated, but there's no guarantee that it'll be one big contiguous chunk. It might be several smaller chunks of a couple of hundred MB each.

If you compile with the LargeAddressAware flag, 64-bit Windows will let you use the full 4GB address space, which should help a bit, but in general,

  • you shouldn't assume that the available memory is contiguous. You should be able to work with multiple smaller allocations rather than a few big ones, and
  • You should compile it as a 64-bit application if you need a lot of memory.

Related questions

c++ what is "pointer = new type" as opposed to "pointer = new type []"?

In many tutorials, the first code samples about dynamic memory start along the lines of: int * pointer; pointer = new int; // version 1 //OR pointer = new int [20]; // version 2 They always proceed to explain how the second version works, but totally avoid talking …

Read More
Allocating struct with variable length array member

I know I can do new char[n] to create an array of n chars. This works even when n is not a compile time constant. But lets say I wanted a size variable followed by n chars: My first …

Read More
new operator for memory allocation on heap

I was looking at the signature of new operator. Which is: void* operator new (std::size_t size) throw (std::bad_alloc); But when we use this operator, we never use a cast. i.e int *arr = new int; So, …

Read More