What is a Buffer Overflow and How Hackers Exploit these Flaws part #1

0000000000001119 <main>:
    1119:    55                       push   rbp
    111a:    48 89 e5                 mov    rbp,rsp
    111d:    c7 45 fc 0f 00 00 00     mov    DWORD PTR [rbp-0x4],0xf
    1124:    b8 00 00 00 00           mov    eax,0x0
    1129:    5d                       pop    rbp
    112a:    c3                       ret    
    112b:    0f 1f 44 00 00           nop    DWORD PTR [rax+rax*1+0x0]

Do you remember what we saw above about rbp? If you already forgot, I kindly suggest you go back to the top! So let's continue and check together what exactly our program does.

• push rbp, put rbp on the stack
• mov rbp, rsp, puts rsp in rbp
• mov DWORD PTR [rbp-0x4], 0xf puts 0xf (15 in hexadecimal) in rbp
• mov eax, 0x0 puts 0 in eax
• pop rbp puts what's at the top of the stack in rbp
• ret quit main

As you can see there is no define variable. We can even say there is no variable name either but ultimately we do have 15 in rbp and it's exactly what we wanted no?

We will try to make this more clear just to be sure that you got it. Let's take this little piece of code:

cat >> ~/Backtest/piece.c << EOL
#include <stdio.h>

int main(void) {
    return 0;
}
EOL

gcc -o piece piece.c
size piece

Now let's see what happens if we modify a little bit of our code by adding a new variable.

cat >> ~/Backtest/piece.c << EOL
#include <stdio.h>

int globalvariable;

int main(void) {
    return 0;
}
EOL

gcc -o piece piece.c
size piece

As you can see bss went from 8 bytes to 12 bytes we can basically and easily consider that our global variable is stored into bss since his value did increase.

Now we will make another test by including a static variable into our main function and see what happens.

cat >> ~/Backtest/piece.c << EOL
#include <stdio.h>

int main(void) {
    static int staticvariable;
    return 0;
}
EOL

gcc -o piece piece.c
size piece

As you can see the variable was not stored in bss like in the previous example but in data which went from 512 bytes to 516 bytes. That looks very interesting no?

We will make a final test to see what could happen if we initialize a global variable.

cat >> ~/Backtest/piece.c << EOL
#include <stdio.h>

int global = 1;

int main(void) {
    return 0;
}
EOL

gcc -o piece piece.c
size piece

We got the same result and do you know why? A global variable, if it is initialized, will be put in data. I think we developed enough about this point for you to understand where and how the variables are stored.

How Does Memory Work?

Here we will not talk about your physical memory but rather RAM and how is it managed by your operating system. A process that runs on a machine requires memory, and in a computer, the amount of memory is limited.

Processes must, therefore, go looking for available memory to be able to work. Let's say several processes are running at the same time. What would happen if two processes wanted to access the same memory area? And if ever a process wrote in a memory area, then another process overwrites this same memory area with its data, then the first process will think of finding its data, but it will find the data of the second process. This could be a very big problem no?

This is where a major feature of your operating system comes to provide a solution to this issue, by allocating to each process a range of virtual memory which is limited to 4GB on 32-bit systems and 8 on 64-bit systems.

Each process will be able to use the memory addresses it needs without worrying about the other processes, the kernel of the operating system will manage to link virtual memory and real memory.

The Stack And The Heap

We will now move on to something very important. The heap can be manipulated by the programmer. This is the part of the memory in which the dynamically allocated memory areas malloc() or calloc() is written.

This memory area does not have a fixed size. It increases and decreases according to what we ask, we can reserve or delete blocks via allocation or release algorithms for future use. The larger the heap size, the larger the memory addresses, and the closer they are to the memory addresses in the stack. The size of the variables in the heap is not limited except the physical memory limit, unlike the stack.

Variables stored in the heap are accessible everywhere in the program with pointers. The stack also has a variable size, but the more its size increases, the more the memory addresses decrease, approaching the top of the heap. The stack grows down. This is where we find the local variables of the functions.

The stack frame of a function is a memory area in the stack in which all the information necessary for calling this function is stored. There are also local variables for the function.

Deepen the Concept of Stack

Let's start with LIFO, which doesn't represent anything complicated since we already see it a bit previously. LIFO stands for Last In, First Out. That is to say that the last thing put on the stack is the first that we will release, we saw it in particular through pop and push.

• We're doing a pop rax; rax is on the stack
• We do a push rbp; we have rax in rbp, rax is removed from the stack

Actually no, it isn't totally like this. rax stays at the same address except that rsp now points to a lower address.

Finally The Buffer Overflow

Finally, we come to the exploitation part. Let's take a small piece of code like the below one.

#include <stdio.h>

int main () {
    char buffer [100];
    int a;
    scanf ("% s", buffer);
}

The code looks completely normal, you must have used the scanf function in your learning of the "C" language. But what if we look at what the stack looks like in this example.

[buffer (100)] [int a] [saved ebp] [saved eip]

You may be wondering what is saved ebp and saved eip? Well, we don't care about "saved ebp", what interests us is "saved eip". Do you remember what eip contains? The address of the next instruction to execute. And if we change this address we could run anything!

But how do you change this value? Well, it's very simple! You see scanf does not check the number of characters it receives! Let's take a piece of code to make it more clear.

#include <stdio.h>

int adminfunction() {
    printf("You successfully jumped on 'adminfunction' ! \n");
}

int vuln() {
    char buffer[10];
    scanf("%s", buffer);
}

int main() {
    vuln()
}

If during the execution of scanf we give a too large value, for example, several "A" this will cause a buffer overflow. The program will, therefore, return a segmentation fault to us. But if we changed the "A" value by a valid address we could jump anywhere or in particular on the adminfunction(). It's done for the theoretical part. We will see the practice in the next article.

Conclusion

In conclusion, buffer overflows remain a persistent challenge for developers worldwide. By adopting secure coding practices, vigilant testing, and continuous education, we can mitigate these risks and fortify our applications against malicious exploits.

Created by

Nicolas C.