Smashing the stack

Segmentation fault

Consider the following program (example1.c):

void function(int a, int b, int c) {
    char buffer1[5];
    char buffer2[10];
}
int main() {
    function(1, 2, 3);
    return 0;
}

Exercise 2.a: compile the program to assembly language like this:

$ gcc -S -o example1.s example1.c

and identify where function is called in the generated assembly code. How are arguments passed to the function? (The answer might vary depending on the compiler and architecture, but the assembly code will tell you!)

Exercise 2.b: compile, execute, and debug the following program:

#include <string.h>

void function(char *str) {
    char buffer[16];
    strcpy(buffer, str);
}

int main() {
    char large_string[256];
    int i;

    for(i = 0; i < 255; i++) {
        large_string[i] = 'A';
    }
    function(large_string);

    return 0;
}

Explain what’s happening and why.

Subverting the control flow

Historically (on 32-bit architectures and with fewer built-in defenses in C compilers), subverting the control flow via stack overflows was pretty easy. You can get an idea of how easy it was by reading the famous tutorial article Smashing The Stack For Fun And Profit by Aleph One (1996).

Nowadays it has become a little more complicated, but it is still very doable (and regularly done, as we have seen in the course examples!).

Exercise 2.c: go through the modern incarnation of Aleph One’s tutorial, namely: Smashing the Stack in the 21st Century by Jon Gjengset (archived copy), reproducing the reported experiments on your machine. (Next week, we will go through some of the defenses that you will have to manually disable to exploit a stack overflow, and discuss what they are and are not good for.)