Stack smashing detected

Question

I am executing my a.out file. After execution the program runs for some time then exits with the message:

**** stack smashing detected ***: ./a.out terminated*
*======= Backtrace: =========*
*/lib/tls/i686/cmov/libc.so.6(__fortify_fail+0x48)Aborted*

What could be the possible reasons for this and how do I rectify it?

Answer 1

Stack Smashing here is actually caused due to a protection mechanism used by gcc to detect buffer overflow errors. For example in the following snippet:

#include <stdio.h>

void func()
{
    char array[10];
    gets(array);
}

int main(int argc, char **argv)
{
    func();
}

The compiler, (in this case gcc) adds protection variables (called canaries) which have known values. An input string of size greater than 10 causes corruption of this variable resulting in SIGABRT to terminate the program.

To get some insight, you can try disabling this protection of gcc using option -fno-stack-protector while compiling. In that case you will get a different error, most likely a segmentation fault as you are trying to access an illegal memory location. Note that -fstack-protector should always be turned on for release builds as it is a security feature.

You can get some information about the point of overflow by running the program with a debugger. Valgrind doesn't work well with stack-related errors, but like a debugger, it may help you pin-point the location and reason for the crash.