Introduction to exploit development - Part 1: Lab setup

Oct 31, 2025

Overview

This mini-series explores the absolute basics of exploit development on Linux systems, targeting both the x86 and ARM architectures. The series is structured as an educational introduction to binary exploitation, covering stack overflows, shellcode development, heap exploitation, and privilege escalation techniques.

Note: This series originated from undergraduate coursework completed in 2019, subsequently adapted and expanded for this blog. While the fundamental concepts remain relevant (memory is still memory, strcpy() still doesn't check bounds), modern exploitation has evolved since then. The material deliberately focuses on foundational techniques in controlled lab environments rather than current real-world attack scenarios. For contemporary resources, see the resources section in the epilogue.

In this first part, we'll configure our lab environment for use in subsequent exploit development posts/exercises. We will configure test environments for both x86 and ARM architectures using QEMU virtualization. For ARM, we will also cover a physical Raspberry Pi setup as an alternative. Both environments require deliberately weakened security settings to facilitate learning - security mitigations that would normally prevent exploitation (or at least make it much harder) will be disabled.

⚠️ Disclaimer

The techniques, tools, and configurations presented in this series are intended exclusively for research and educational purposes. Exploitation techniques should only be applied in controlled lab environments or with explicit authorization. I am not responsible for any misuse of the information provided in this series.

Lab environment setup

x86 environment: QEMU virtual machine

For x86 exploit development, we use QEMU to emulate a 32-bit i386 system. The 32-bit architecture simplifies initial learning - memory addresses are shorter, stack layouts are more straightforward, and exploit payloads are easier to construct compared to x86_64.

QEMU provides full system emulation with decent performance and debugging capabilities.

Target system specifications:

  • OS: Debian 12 (Bookworm) i386
  • Architecture: 32-bit x86 (i386)
  • Virtualization platform: QEMU
  • Emulated hardware: Standard PC (i440FX chipset)

Installing QEMU

Install QEMU using your system package manager, e.g., on Arch:

# Arch Linux
sudo pacman -S qemu-base qemu-system-x86

Creating the x86 virtual machine

Download a Debian 12 i386 netinst ISO from the official Debian archives.

✏️ Debian 12 is the last release that supports i386, don't waste time trying to find a more recent i386 version of Debian

Create a virtual disk image (20 GB is sufficient):

qemu-img create -f qcow2 debian-i386-exploit-dev.qcow2 20G

Extract the Linux kernel and the initrd for the text-based installer:

sudo mount -o loop,ro debian-12.10.0-i386-netinst.iso /mnt
cp /mnt/install.386/{initrd.gz,vmlinuz} ./
sudo umount /mnt

Boot the installer ISO:

qemu-system-i386 \
    -m 2G -cpu max \
    -nographic -serial mon:stdio \
    -net nic -net user \
    -hda debian-i386-exploit-dev.qcow2 \
    -kernel vmlinuz -initrd initrd.gz \
    -append "console=ttyS0,115200n8" \
    -boot d \
    -cdrom debian-12.10.0-i386-netinst.iso

Follow the Debian installer to perform a minimal installation. During installation:

  • Select no desktop environment (we only need SSH and standard system utilities)
  • Configure network settings for SSH access
  • Install the GRUB bootloader to the virtual disk

After installation completes, shut down the VM.

Running the x86 VM

After installation, start the system with user-mode networking and SSH port forwarding:

qemu-system-i386 \
    -m 2G -cpu max \
    -nographic -serial mon:stdio \
    -net nic -net user,hostfwd=tcp::2222-:22 \
    -hda debian-i386-exploit-dev.qcow2

The VM console will display on your terminal. Log in with your configured credentials.

SSH access to the x86 VM

With port forwarding configured, connect via SSH from the host:

ssh -p 2222 user@localhost

This provides a more convenient interface than the serial console.

Installing development tools on x86 VM

Inside the VM, switch to root then install necessary packages:

su
apt update
apt install build-essential gdb python3 vim sudo

ARM environment: QEMU virtual machine

For ARM exploit development, we use QEMU to emulate an ARMv7 system. This provides a consistent, reproducible environment without requiring physical ARM hardware.

Target system specifications:

  • OS: Debian 12 (Bookworm) armhf
  • Architecture: 32-bit ARMv7 (armhf)
  • Virtualization platform: QEMU
  • Emulated machine: virt (generic ARM virtual machine)

Installing QEMU for ARM

Install the ARM system emulator using your system package manager, e.g., on Arch:

sudo pacman -S qemu-system-arm

Creating the ARM virtual machine

Download a Debian 12 armhf netinst ISO from the official Debian archives.

Create a virtual disk image:

qemu-img create -f qcow2 debian-armhf-exploit-dev.qcow2 20G

Extract the Linux kernel and initrd for the text-based installer:

sudo mount -o loop,ro debian-12.10.0-armhf-netinst.iso /mnt
cp /mnt/install.ahf/{initrd.gz,vmlinuz} ./
sudo umount /mnt

Boot the installer ISO:

qemu-system-arm \
    -m 2G -M virt -cpu cortex-a15 \
    -nographic -serial mon:stdio \
    -netdev user,id=net0 -device virtio-net-device,netdev=net0 \
    -drive if=none,file=debian-armhf-exploit-dev.qcow2,format=qcow2,id=hd \
    -device virtio-blk-device,drive=hd \
    -kernel vmlinuz -initrd initrd.gz \
    -append "console=ttyAMA0" \
    -drive if=none,file=debian-12.10.0-armhf-netinst.iso,id=cdrom,media=cdrom,readonly=on \
    -device virtio-blk-device,drive=cdrom

Follow the Debian installer to perform a minimal installation. During installation:

  • Select no desktop environment (we only need SSH and standard system utilities)
  • Configure network settings for SSH access
  • Install the GRUB bootloader to the virtual disk

After installation completes, shut down the VM.

Running the ARM VM

After installation, extract the kernel from the installed system:

# Mount the qcow2 image
sudo modprobe nbd max_part=8
sudo qemu-nbd --connect=/dev/nbd0 debian-armhf-exploit-dev.qcow2
sudo mount /dev/nbd0p1 /mnt  # Adjust partition number as needed

# Copy kernel and initrd
sudo cp /mnt/boot/vmlinuz ./vmlinuz
sudo cp /mnt/boot/initrd.img ./initrd.gz

# Unmount
sudo umount /mnt
sudo qemu-nbd --disconnect /dev/nbd0
sudo modprobe -r nbd

Finally, start the system with user-mode networking and SSH port forwarding:

qemu-system-arm \
    -m 2G -M virt -cpu cortex-a15 \
    -nographic -serial mon:stdio \
    -netdev user,id=net0,hostfwd=tcp::2223-:22 -device virtio-net-device,netdev=net0 \
    -kernel vmlinuz -initrd initrd.gz \
    -append "console=ttyAMA0" \
    -drive if=none,file=debian-armhf-exploit-dev.qcow2,format=qcow2,id=hd \
    -device virtio-blk-device,drive=hd

The VM console will display on your terminal. Log in with your configured credentials.

SSH access to the ARM VM

Connect via SSH with port forwarding:

ssh -p 2223 user@localhost

Installing development tools on ARM VM

Inside the VM, install necessary packages:

su
apt update
apt install build-essential gdb python3 vim sudo

ARM environment: Raspberry Pi (alternative)

As an alternative to QEMU emulation, you can use a physical Raspberry Pi to experience running against a real ARM environment. This is also more fun, obviously.

Hardware specifications:

  • Hardware: Raspberry Pi (Model 2B or later recommended for ARMv7)
  • OS: Raspberry Pi OS Lite (based on Debian)
  • Architecture: ARMv7 or ARMv8 (32-bit mode)

Installing Raspberry Pi OS

Download Raspberry Pi OS Lite from the official website. Use the Raspberry Pi Imager or dd to write the image to a microSD card:

sudo dd if=raspios-lite.img of=/dev/sdc bs=4M status=progress conv=fsync

Note: Replace /dev/sdc with your actual microSD card device. Verify the device path with lsblk to avoid nuking your host system.

Enabling SSH

Before first boot, mount the boot partition and create an empty file named ssh to enable SSH:

sudo mount /dev/sdc1 /mnt
sudo touch /mnt/ssh
sudo umount /mnt

Insert the microSD card into the Raspberry Pi and power it on. The device will be accessible via SSH on your network.

Installing development tools on Raspberry Pi

Connect via SSH and install necessary packages:

sudo apt update
sudo apt install build-essential gdb python3 vim sudo

Disabling security mitigations

Modern Linux systems implement multiple layers of security to prevent exploitation. For practical/educational purposes, we will disable most protections to make exploitation mechanics easier and more clear. Note however, that normally these mitigations would be enabled by default and remain enabled.

Address Space Layout Randomization (ASLR)

ASLR randomizes the memory layout of processes, making it difficult for attackers to predict the locations of stack frames, heap allocations, and libraries. By randomizing addresses on each execution, ASLR breaks exploits that rely on hardcoded memory addresses.

To disable ASLR, we modify the kernel parameter kernel.randomize_va_space. This parameter accepts three values:

  • 0: No randomization (ASLR disabled)
  • 1: Conservative randomization (stack, libraries, heap randomized; text segment fixed)
  • 2: Full randomization (all memory regions randomized)

Temporary disabling (lost on reboot):

sudo /sbin/sysctl kernel.randomize_va_space=0

Permanent disabling via sysctl configuration:

echo "kernel.randomize_va_space = 0" | sudo tee /etc/sysctl.d/01-disable-aslr.conf
sudo /sbin/sysctl -p /etc/sysctl.d/01-disable-aslr.conf

Verify the setting:

sudo /sbin/sysctl kernel.randomize_va_space
# Output: kernel.randomize_va_space = 0

This configuration applies to all environments (x86 QEMU, ARM QEMU, and Raspberry Pi).

Compiler security features

Modern C compilers implement several security features by default, for example, GCC provides the following:

  1. Stack canaries (-fstack-protector): Places a random value (canary) before the saved return address on the stack. If a buffer overflow overwrites the return address, it will also corrupt the canary, which is checked before function return. Canary corruption triggers program termination.

  2. Non-executable stack (NX bit, -z noexecstack): Marks the stack memory region as non-executable, preventing shellcode execution from stack buffers.

  3. Stack alignment enforcement: Modern calling conventions enforce stack alignment requirements (16-byte alignment on x86-64, for example), which can complicate exploit payloads.

To compile vulnerable binaries suitable for learning, we must explicitly disable these protections:

# x86
gcc -m32 \
    -fno-stack-protector \
    -z execstack \
    -mpreferred-stack-boundary=2 \
    -no-pie \
    -ggdb \
    -o vulnerable_program \
    vulnerable_program.c

# ARM
gcc -fno-stack-protector \
    -z execstack \
    -no-pie \
    -ggdb \
    -o vulnerable_program \
    vulnerable_program.c

Flag explanations:

  • -m32: Compile for 32-bit x86 (useful even on 64-bit hosts for simpler exploits)
  • -fno-stack-protector: Disable stack canaries
  • -z execstack: Mark stack memory as executable (allows shellcode execution)
  • -mpreferred-stack-boundary=2: Reduce stack alignment to 4 bytes (2^2), simplifying stack layout
  • -no-pie: Disable Position-Independent Executable, ensuring consistent load addresses
  • -ggdb: Include debugging symbols for GDB

In future posts, we'll look into techniques to exploit binaries with some of these security features still enabled. For now, make sure you compile each binary with the flags above to start.

Core dumps

Core dumps capture the memory state of a process when it crashes, providing valuable debugging information for exploit development. By default, many systems disable core dumps or limit their size.

Enable unlimited core dumps:

ulimit -c unlimited

To make this persistent across sessions, add it to your shell profile (~/.bashrc or ~/.profile):

echo "ulimit -c unlimited" >> ~/.bashrc

Core dumps will be written to the current working directory (or the location specified by /proc/sys/kernel/core_pattern).

Testing the environment

To verify the lab environment is configured correctly, let's compile and execute a deliberately vulnerable test program.

test.c

#include <stdio.h>
#include <string.h>

void vulnerable_function(char *input) {
    char buffer[64];
    strcpy(buffer, input);  // Unsafe: no bounds checking
    printf("You entered: %s\n", buffer);
}

int main(int argc, char *argv[]) {
    if (argc != 2) {
        printf("Usage: %s <input>\n", argv[0]);
        return 1;
    }

    vulnerable_function(argv[1]);
    printf("Exiting normally.\n");
    return 0;
}

This program contains a classic stack-based buffer overflow vulnerability. The strcpy() function copies user input into a fixed-size buffer without bounds checking. If the input exceeds 64 bytes, the overflow will corrupt adjacent stack memory, including the saved return address.

Compilation:

# x86
gcc -m32 -fno-stack-protector -z execstack -mpreferred-stack-boundary=2 \
    -no-pie -ggdb -o test test.c

# ARM
gcc -fno-stack-protector -z execstack -no-pie -ggdb -o test test.c

Normal execution:

./test "Hello, World!"
# Output:
# You entered: Hello, World!
# Exiting normally.

Triggering the overflow:

./test $(python3 -c 'print("A" * 100)')
# Output:
# You entered: AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA...
# Segmentation fault (core dumped)

The segmentation fault indicates the overflow has corrupted the return address, causing the program to jump to an invalid memory location (likely 0x41414141, the ASCII representation of "AAAA"). The presence of "core dumped" confirms our core dump configuration is working.

Examine the core dump with GDB:

gdb ./test core
# Inside GDB:
(gdb) info registers
(gdb) x/20x $esp
(gdb) bt

This should reveal that the instruction pointer (EIP on x86) has been overwritten with 0x41414141, confirming successful control flow hijacking.

Raspberry Pi UART serial access (optional)

While not strictly necessary for exploit development, UART (Universal Asynchronous Receiver-Transmitter) serial access to the Raspberry Pi provides a useful debugging interface independent of network connectivity. Many embedded devices expose UART pins for debugging, and understanding serial communication is valuable for hardware security research.

Enabling UART on Raspberry Pi

Before first boot, mount the microSD card's boot partition and enable UART in the configuration file:

sudo mount /dev/sdc1 /mnt
echo "enable_uart=1" >> /mnt/config.txt
sudo umount /mnt

Connecting via serial-to-USB adapter

UART requires a serial-to-USB adapter (e.g., FTDI or CP2102-based adapters). Connect the adapter to the Raspberry Pi's GPIO pins:

  • TX (adapter) → RX (GPIO 15, Pin 10)
  • RX (adapter) → TX (GPIO 14, Pin 8)
  • GND (adapter) → GND (Pin 6)

Do not connect the VCC pin - the Pi should be powered separately via its standard power supply.

Accessing the serial console

Once connected, use screen (or minicom) to access the serial console:

sudo screen /dev/ttyUSB0 115200,cs8,-parenb,-cstopb,-hupcl

Parameter explanation:

  • 115200: Baud rate (bits per second)
  • cs8: 8 data bits
  • -parenb: No parity bit
  • -cstopb: 1 stop bit
  • -hupcl: No modem control signals

You should now have a TTY console on the Raspberry Pi, accessible without network connectivity or HDMI output. This is particularly useful for debugging boot issues or network configuration problems.

To exit screen, press Ctrl+A then K (confirm with Y).

Verification checklist

Before proceeding to exploit development, verify the following:

  • Virtual machine(s) are running and accessible via SSH
  • ASLR is disabled (verify with sysctl kernel.randomize_va_space)
  • Core dumps are enabled (verify with ulimit -c)
  • GCC can compile with -fno-stack-protector and -z execstack
  • Test program crashes with buffer overflow input
  • Core dumps are generated and readable with GDB
  • (Raspberry Pi only) UART serial access works if configured

Series roadmap

The remaining parts of this series cover:

  • Part 2: Stack overflows: Understanding memory layout, stack frames, and the mechanics of buffer overflow vulnerabilities that enable control flow hijacking
  • Part 3: Shellcode: Writing position-independent shellcode, NOP sleds, and Return-Oriented Programming techniques for both architectures
  • Part 4: Heap overflows: Heap memory allocation mechanics, Global Offset Table manipulation, and function pointer overwriting

Each part builds progressively on previous concepts, with complete source code and detailed explanations of underlying mechanics.

References

https://betweenlayers.io/posts/feed.xml