When and why will a compiler initialise memory to 0xCD, 0xDD, etc. on malloc/free/new/delete?
I know that the compiler will sometimes initialize memory with certain patterns such as 0xCD and 0xDD. What I want to know is when and why this happens.
When
Is this specific to the compiler used?
Do malloc/new and free/delete work in the same way with regard to this?
Is it platform specific?
Will it occur on other operating systems, such as Linux or VxWorks?
Why
My understanding is this only occurs in Win32 debug configuration, and it is used to detect memory overruns and to help the compiler catch exceptions.
Can you give any practical examples as to how this initialization is useful?
I remember reading something (maybe in Code Complete 2) saying that it is good to initialize memory to a known pattern when allocating it, and certain patterns will trigger interrupts in Win32 which will result in exceptions showing in the debugger.
How portable is this?
Answer
A quick summary of what Microsoft's compilers use for various bits of unowned/uninitialized memory when compiled for debug mode (support may vary by compiler version):
Value Name Description
------ -------- -------------------------
0xCD Clean Memory Allocated memory via malloc or new but never
written by the application.
0xDD Dead Memory Memory that has been released with delete or free.
It is used to detect writing through dangling pointers.
0xED or Aligned Fence 'No man's land' for aligned allocations. Using a
0xBD different value here than 0xFD allows the runtime
to detect not only writing outside the allocation,
but to also identify mixing alignment-specific
allocation/deallocation routines with the regular
ones.
0xFD Fence Memory Also known as "no mans land." This is used to wrap
the allocated memory (surrounding it with a fence)
and is used to detect indexing arrays out of
bounds or other accesses (especially writes) past
the end (or start) of an allocated block.
0xFD or Buffer slack Used to fill slack space in some memory buffers
0xFE (unused parts of `std::string` or the user buffer
passed to `fread()`). 0xFD is used in VS 2005 (maybe
some prior versions, too), 0xFE is used in VS 2008
and later.
0xCC When the code is compiled with the /GZ option,
uninitialized variables are automatically assigned
to this value (at byte level).
// the following magic values are done by the OS, not the C runtime:
0xAB (Allocated Block?) Memory allocated by LocalAlloc().
0xBAADF00D Bad Food Memory allocated by LocalAlloc() with LMEM_FIXED,but
not yet written to.
0xFEEEFEEE OS fill heap memory, which was marked for usage,
but wasn't allocated by HeapAlloc() or LocalAlloc().
Or that memory just has been freed by HeapFree().
Disclaimer: the table is from some notes I have lying around - they may not be 100% correct (or coherent).
Many of these values are defined in vc/crt/src/dbgheap.c:
/*
* The following values are non-zero, constant, odd, large, and atypical
* Non-zero values help find bugs assuming zero filled data.
* Constant values are good, so that memory filling is deterministic
* (to help make bugs reproducible). Of course, it is bad if
* the constant filling of weird values masks a bug.
* Mathematically odd numbers are good for finding bugs assuming a cleared
* lower bit.
* Large numbers (byte values at least) are less typical and are good
* at finding bad addresses.
* Atypical values (i.e. not too often) are good since they typically
* cause early detection in code.
* For the case of no man's land and free blocks, if you store to any
* of these locations, the memory integrity checker will detect it.
*
* _bAlignLandFill has been changed from 0xBD to 0xED, to ensure that
* 4 bytes of that (0xEDEDEDED) would give an inaccessible address under 3gb.
*/
static unsigned char _bNoMansLandFill = 0xFD; /* fill no-man's land with this */
static unsigned char _bAlignLandFill = 0xED; /* fill no-man's land for aligned routines */
static unsigned char _bDeadLandFill = 0xDD; /* fill free objects with this */
static unsigned char _bCleanLandFill = 0xCD; /* fill new objects with this */
There are also a few times where the debug runtime will fill buffers (or parts of buffers) with a known value, for example, the 'slack' space in std::string's allocation or the buffer passed to fread(). Those cases use a value given the name _SECURECRT_FILL_BUFFER_PATTERN (defined in crtdefs.h). I'm not sure exactly when it was introduced, but it was in the debug runtime by at least VS 2005 (VC++8).
Initially, the value used to fill these buffers was 0xFD - the same value used for no man's land. However, in VS 2008 (VC++9) the value was changed to 0xFE. I assume that's because there could be situations where the fill operation would run past the end of the buffer, for example, if the caller passed in a buffer size that was too large to fread(). In that case, the value 0xFD might not trigger detecting this overrun since if the buffer size were too large by just one, the fill value would be the same as the no man's land value used to initialize that canary. No change in no man's land means the overrun wouldn't be noticed.
So the fill value was changed in VS 2008 so that such a case would change the no man's land canary, resulting in the detection of the problem by the runtime.
As others have noted, one of the key properties of these values is that if a pointer variable with one of these values is de-referenced, it will result in an access violation, since on a standard 32-bit Windows configuration, user mode addresses will not go higher than 0x7fffffff.