What exactly does _malloc do in assembly?

user3089458 picture user3089458 · Dec 11, 2013 · Viewed 23.1k times · Source
public main
main proc near
push    ebp
mov     ebp, esp
and     esp, 0FFFFFFF0h
sub     esp, 30h
mov     dword ptr [esp], 8 ; size
call    _malloc
mov     [esp+2Ch], eax
mov     dword ptr [esp+4], 4
mov     eax, [esp+2Ch]
mov     [esp], eax
call    __start

The code above represents a portion of a large project I am working on. I am trying to reverse this code into C equivalent but I am having difficulty understanding how malloc works.

I am figuring 8 bytes would be the size of the memory being allocated; however, I am not sure about this line.

mov      eax, [esp+2ch] 

What does malloc do to eax?

Furthermore would this be equivalent C code?

int main(void)
{
int *ptr1;
ptr1 = (int *)malloc(sizeof(8));
*ptr1 = 4;
__start(*ptr1);

Answer

Sajad Karuthedath picture Sajad Karuthedath · Dec 11, 2013

The function malloc() will allocate a block of memory that is size bytes large. If the requested memory can be allocated a pointer is returned to the beginning of the memory block.

Note: the content of the received block of memory is not initialized.

Syntax of malloc():

void *malloc ( size_t size );

Parameters:

Size of the memory block in bytes.

Return value:

If the request is successful then a pointer to the memory block is returned. If the function failed to allocate the requested block of memory, a NULL is returned, NULL may also be returned by a successful call to malloc() with a size of zero.

As stated in this CS 301 lecture by Dr. Lawlor:

Calling Malloc from Assembly Language

It's a pretty straightforward function: pass the number of BYTES you want as the only parameter, in rdi. "call malloc." You'll get back a pointer to the allocated bytes returned in rax. To clean up the space afterwards, copy the pointer over to rdi, and "call free" (I'm leaving off the free below, because you need the stack to do that properly).

Here's a complete example of assembly memory access. I call malloc to get 40 bytes of space. malloc returns the starting address of this space in rax (the 64-bit version of eax). That is, the rax register is acting like a pointer. I can then read and write from the pointed-to memory using the usual assembly bracket syntax:

mov edi, 40; malloc's first (and only) parameter: number of bytes to allocate
extern malloc
call malloc
; on return, rax points to our newly-allocated memory
mov ecx,7; set up a constant
mov [rax],ecx; write it into memory
mov edx,[rax]; read it back from memory
mov eax,edx; copy into return value register
ret

Rather than copy via the ecx register, you can specify you want a 32-bit memory write and read using "DWORD" in front of the brackets, like this:

mov edi, 40; malloc's first (and only) parameter: number of bytes to allocate
extern malloc
call malloc
; on return, rax points to our newly-allocated memory
mov DWORD [rax],7; write constant into memory
mov eax,DWORD [rax]; read it back from memory
ret

for malloc in assembly language..see this link malloc