How are multi-threaded programs implemented in Assembly language

The idea of “threads” feels like a high-level concept — something you’d manage with a library call in C or a Thread object in a language like Java. But underneath every one of those abstractions is raw assembly: register saves, stack manipulation, atomic instructions, and trap instructions into the kernel. I remember the moment this really landed for me — realizing that a “thread” isn’t some magical OS entity floating in space, it’s just a saved set of register values and a stack pointer that the CPU switches to and from.

In this post, I want to unpack how multi-threading actually gets implemented at the assembly level: how threads are created via system calls, how the CPU context-switches between them, how synchronization primitives like mutexes and atomics are built from individual instructions, and how memory models and cache coherence come into play. This is a deeper topic than the other two flags/instructions I’ve written about, so I’ll move from fundamentals to genuinely advanced territory.

Table of Contents

1. What Does “Multi-Threaded” Mean at the Assembly Level?
2. Processes vs Threads: The Memory Picture
3. Creating a Thread: The Syscall Path
4. Context Switching Explained with Registers and Stack
5. x86-64 Assembly Example: Manual Thread Creation
6. ARM64 Assembly Example: Manual Thread Creation
7. Synchronization Primitives in Raw Assembly
8. Atomic Instructions and the LOCK Prefix
9. Memory Models and Memory Barriers
10. Internal Working Process (with Diagram)
11. Practical Use Cases
12. OS Scheduler Interaction
13. Debugging Multi-Threaded Assembly
14. Optimization and Performance Considerations
15. Comparison: Mutex vs Spinlock vs Atomic Operations
16. Best Practices
17. Common Mistakes
18. FAQs
19. Summary and Key Takeaways
20. References

1. What Does “Multi-Threaded” Mean at the Assembly Level?

At the hardware level, a CPU core executes one instruction stream at a time (ignoring SMT/hyperthreading for a moment, which is a separate topic). “Multiple threads” running concurrently is an illusion created by the operating system rapidly switching which instruction stream a core is executing — saving one thread’s complete register state, loading another’s, and jumping to where that thread left off. Everything a high-level thread library does — pthread_create, std::thread, Java’s Thread — ultimately reduces to:

1. Allocating a new stack region in memory.
2. Making a system call to ask the kernel to create a new schedulable execution context (clone on Linux).
3. Populating that context with an initial instruction pointer and stack pointer.
4. Letting the OS scheduler decide when that context actually runs on a core.

2. Processes vs Threads: The Memory Picture

The core distinction between a process and a thread, at the assembly/OS level, is what gets duplicated versus shared:

This is the essential insight: threads share almost everything about the address space except the stack and the register/instruction-pointer state. That’s exactly why race conditions on shared global/heap data are a thread problem but not really a “process” problem in the same way.

3. Creating a Thread: The Syscall Path

On Linux, threads are created via the clone() system call, which is a generalized version of fork() that lets you specify exactly which resources to share via flags like CLONE_VM (share memory), CLONE_FS (share filesystem info), CLONE_FILES (share file descriptor table), and CLONE_THREAD (join the same thread group).

; x86-64 Linux: creating a new thread via clone()
; (simplified — real pthread implementations do more setup)

section .bss
    stack_size equ 1048576         ; 1 MB stack for the new thread
    thread_stack resb stack_size

section .text
global _start

thread_func:
    ; This code runs in the new thread
    mov rax, 1              ; sys_write
    mov rdi, 1
    mov rsi, msg
    mov rdx, msglen
    syscall

    mov rax, 60              ; sys_exit for this thread (exit just the thread ideally uses exit_group carefully)
    xor rdi, rdi
    syscall

_start:
    mov rax, 56                              ; sys_clone
    mov rdi, 0x00010F00                      ; flags: CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD
    lea rsi, [thread_stack + stack_size]      ; new stack pointer (grows down)
    xor rdx, rdx
    xor r10, r10
    syscall

    cmp rax, 0
    je  thread_func           ; child (new thread) jumps to thread_func

    ; parent continues here
    mov rax, 60
    xor rdi, rdi
    syscall

section .data
msg db "Hello from a manually created thread!", 0xA
msglen equ $ - msg

This is a simplified illustration — real-world thread creation (like glibc’s pthread_create) involves more careful setup of thread-local storage, signal masks, and the thread control block, but the core mechanism is exactly this: a clone syscall with a new stack pointer and the right sharing flags.

4. Context Switching Explained with Registers and Stack

When the OS scheduler decides to switch from Thread A to Thread B on a core, it performs what’s called a context switch. In assembly terms, this means:

1. Push/save all of Thread A’s general-purpose registers, instruction pointer, and flags onto Thread A’s kernel stack (or a dedicated save area).
2. Switch the stack pointer register to Thread B’s saved kernel stack.
3. Pop/restore Thread B’s previously saved registers.
4. Jump (via iret/sysret/eret) to Thread B’s saved instruction pointer.

This is why context switches, while fast, are not free — every switch involves a real chunk of register save/restore work plus potential cache and TLB (Translation Lookaside Buffer) disruption, since the new thread might touch completely different memory regions.

5. x86-64 Assembly Example: Manual Thread Creation

Here’s a minimal, low-level illustration of what a hand-rolled context switch routine looks like (this is close to what you’d see inside a cooperative user-space threading library or a kernel’s own switch routine):

; Simplified cooperative context switch (x86-64)
; void switch_context(void** old_sp, void* new_sp)

switch_context:
    ; Save callee-saved registers of the current thread
    push rbp
    push rbx
    push r12
    push r13
    push r14
    push r15

    ; Save current stack pointer into *old_sp
    mov [rdi], rsp

    ; Load the new thread's stack pointer
    mov rsp, rsi

    ; Restore callee-saved registers of the new thread
    pop r15
    pop r14
    pop r13
    pop r12
    pop rbx
    pop rbp

    ret        ; returns into the new thread's saved return address

This pattern — save registers, swap stack pointers, restore registers, return — is the essence of every user-space cooperative threading/coroutine library (like fibers, green threads, or Go’s early goroutine scheduler internals).

6. ARM64 Assembly Example: Manual Thread Creation

// Simplified cooperative context switch (ARM64/AArch64)
// void switch_context(void** old_sp, void* new_sp)

switch_context:
    // Save callee-saved registers
    stp x19, x20, [sp, #-16]!
    stp x21, x22, [sp, #-16]!
    stp x23, x24, [sp, #-16]!
    stp x29, x30, [sp, #-16]!    // frame pointer + link register

    // Save current SP into *old_sp
    mov x2, sp
    str x2, [x0]

    // Load new thread's SP
    mov sp, x1

    // Restore callee-saved registers
    ldp x29, x30, [sp], #16
    ldp x23, x24, [sp], #16
    ldp x21, x22, [sp], #16
    ldp x19, x20, [sp], #16

    ret

Same idea, ARM64 conventions: stp/ldp (store/load pair) instructions and the x30 link register standing in for x86’s return address on the stack.

7. Synchronization Primitives in Raw Assembly

Threads sharing memory need a way to coordinate access, or you get race conditions. The most fundamental primitive is a spinlock, built directly from an atomic exchange instruction:

; x86-64 spinlock acquire/release using XCHG (implicitly atomic)
; lock_var: dd 0   -> 0 = unlocked, 1 = locked

acquire_lock:
    mov eax, 1
.retry:
    xchg eax, [lock_var]   ; XCHG is always atomic on x86, no LOCK prefix needed
    test eax, eax
    jnz .spin               ; if it was already 1, spin
    ret
.spin:
    pause                   ; hint to the CPU this is a spin-wait loop
    mov eax, 1
    jmp .retry

release_lock:
    mov dword [lock_var], 0
    ret

The pause instruction here is a performance hint — it tells the CPU this is a busy-wait loop, which reduces power consumption and avoids a memory-order violation penalty on some microarchitectures.

8. Atomic Instructions and the LOCK Prefix

On x86/x86-64, ordinary instructions like ADD or INC are not atomic across cores unless prefixed with LOCK, which asserts a bus/cache lock for the duration of the operation:

lock inc dword [counter]        ; atomic increment, safe across threads/cores
lock cmpxchg [shared_var], ebx  ; atomic compare-and-swap
lock xadd [counter], eax        ; atomic fetch-and-add

CMPXCHG (compare-and-exchange) with the LOCK prefix is the building block behind most modern lock-free data structures — it’s the assembly-level equivalent of the “compare-and-swap” (CAS) primitive used in higher-level lock-free programming.

ARM uses a different but conceptually related mechanism: load-exclusive/store-exclusive pairs (LDXR/STXR on AArch64, LDREX/STREX on AArch32):

// ARM64 atomic increment using load/store-exclusive
retry:
    LDXR    X1, [X0]        // load exclusive
    ADD     X1, X1, #1
    STXR    W2, X1, [X0]    // store exclusive, W2 = 0 if succeeded
    CBNZ    W2, retry        // retry if the store failed (another core interfered)

Newer ARM cores also support dedicated atomic instructions like LDADD (part of the ARMv8.1 Large System Extensions), which behave more like x86’s LOCK XADD.

9. Memory Models and Memory Barriers

Even with atomic instructions, the order in which memory operations become visible to other cores isn’t guaranteed unless you explicitly enforce it. x86 has a relatively strong memory model (mostly program-order preserving for normal loads/stores), while ARM has a notably weaker, more relaxed memory model — meaning ARM code needs explicit memory barrier instructions far more often.

; x86-64 memory fences
mfence   ; full memory fence (serializes all loads/stores)
sfence   ; store fence
lfence   ; load fence

; ARM64 memory barriers
DMB SY   ; Data Memory Barrier, full system
DSB SY   ; Data Synchronization Barrier
ISB      ; Instruction Synchronization Barrier

This is one of the biggest “gotchas” for anyone porting hand-written concurrent assembly from x86 to ARM: code that “just works” on x86 due to its stronger ordering guarantees can produce genuine race conditions on ARM without explicit barriers.

10. Internal Working Process (With Diagram)

Here’s how I visualize the full life cycle of two threads being scheduled on a single core:

flowchart TD
    A[Thread A running on Core] --> B[Timer Interrupt or Syscall Trap Fires]
    B --> C[Scheduler Invoked in Kernel]
    C --> D[Save Thread A registers, SP, PC to its Task Struct]
    D --> E[Scheduler Picks Next Runnable Thread: Thread B]
    E --> F[Load Thread B registers, SP, PC from its Task Struct]
    F --> G[Restore Address Space if different Process]
    G --> H[Return to User Mode via iret/sysret/eret]
    H --> I[Thread B Resumes Execution]

The whole illusion of “simultaneous” threads on a single core is just this loop happening many times per second (modern schedulers commonly use time slices in the single-digit milliseconds), fast enough that it feels continuous to a human.

11. Practical Use Cases

• Operating system kernels — schedulers, interrupt handlers, and low-level synchronization primitives are all hand-written in assembly (or very close to it, in C with inline assembly) for performance and precise hardware control.
• High-performance runtime libraries — coroutine/fiber libraries (like Boost.Context or Go’s early runtime) hand-roll context-switch routines in assembly for speed.
• Lock-free data structures — high-throughput queues and allocators in systems like databases and game engines rely on hand-tuned atomic instructions rather than higher-level mutexes.
• Embedded/RTOS threading — small real-time operating systems often implement their entire task-switching mechanism directly in assembly due to lack of an underlying general-purpose OS.

12. OS Scheduler Interaction

The OS scheduler decides which thread gets to run and when, but the actual mechanics of switching are pure assembly — a small routine (often called context_switch or __switch_to in kernel source) that does exactly the register-save/stack-swap/register-restore dance shown earlier. On Linux specifically, this lives in architecture-specific files like arch/x86/entry/entry_64.S and arch/arm64/kernel/entry.S, which are literally hand-written assembly files in the kernel source tree.

13. Debugging Multi-Threaded Assembly

Debugging concurrency bugs at the assembly level is genuinely hard because the bugs are often timing-dependent and disappear under a debugger’s slower execution (the classic “Heisenbug”). Tools that help:

(gdb) info threads             ; list all threads in the process
(gdb) thread 2                  ; switch to inspecting thread 2
(gdb) thread apply all bt       ; backtrace of every thread at once

For lock-free/atomic code specifically, tools like Intel’s Inspector or the ThreadSanitizer (though usually applied at the C/C++ level, not raw assembly) can catch data races that are otherwise nearly impossible to reproduce reliably.

14. Optimization and Performance Considerations

• Spinlocks vs blocking locks — spinlocks avoid the cost of a full context switch (and trap into the kernel) but waste CPU cycles busy-waiting; they’re only appropriate when the expected wait time is shorter than a context switch would cost.
• False sharing — when two threads’ independent variables happen to sit on the same cache line, both cores’ caches keep invalidating each other’s copies even though there’s no real data dependency. Padding shared structures to cache-line size (typically 64 bytes) avoids this.
• Cache coherence traffic — every atomic instruction (LOCK-prefixed on x86, exclusive-pair on ARM) generates cache coherence protocol traffic between cores, which is why lock-free code isn’t automatically “free” — it just trades blocking overhead for coherence traffic overhead.

15. Comparison: Mutex vs Spinlock vs Atomic Operations

16. Best Practices

• Prefer atomic instructions over full locks for simple counters and flags — they avoid both kernel round-trips and busy-waiting entirely.
• Always insert appropriate memory barriers on ARM when porting concurrent code from x86, since ARM’s weaker memory model won’t save you the way x86’s stronger model does.
• Pad frequently-updated shared variables to avoid false sharing across cache lines.
• Use pause/equivalent hint instructions inside spin loops to reduce power draw and avoid pipeline stalls.

17. Common Mistakes

• Assuming x86-style memory ordering guarantees apply on ARM — this is one of the most common sources of “works on x86, breaks on ARM” concurrency bugs.
• Forgetting the LOCK prefix on x86 read-modify-write instructions, resulting in operations that look atomic in single-threaded testing but aren’t under real concurrency.
• Using a spinlock for a critical section that can block for a long time, wasting enormous amounts of CPU time better spent context-switching to a blocking mutex.
• Not accounting for the fact that clone()‘s flags must be set correctly (CLONE_VM, CLONE_FS, CLONE_FILES, CLONE_THREAD) — getting this wrong silently creates something closer to a new process than a true thread.

18. FAQs

Q: Do CPUs have a native concept of “threads”? Not really — a CPU core just executes whichever instruction stream it’s currently pointed at. “Threads” are an operating system-level abstraction built from register save/restore and stack switching, though CPUs do provide simultaneous multithreading (SMT/hyperthreading) as a hardware feature that lets one core execute two instruction streams concurrently by duplicating some pipeline resources.

Q: What’s the difference between a spinlock and a mutex at the assembly level? A spinlock is just a tight loop around an atomic instruction (like XCHG or CMPXCHG) with no kernel involvement. A mutex, once contended, typically makes a system call (like Linux’s futex) to let the kernel put the waiting thread to sleep instead of burning CPU cycles.

Q: Why does ARM need more memory barriers than x86? x86 has a relatively strong memory ordering model by default, while ARM’s architecture is deliberately weaker/more relaxed to allow more hardware optimization freedom, which means ARM code must be more explicit about ordering requirements using barrier instructions.

Q: Can you write a full thread scheduler purely in assembly? Yes, and many real operating system kernels do exactly this for the architecture-specific low-level switching routines, even if the higher-level scheduling policy is written in C.

19. Summary and Key Takeaways

Multi-threading, when you get all the way down to the assembly level, turns out to be a beautifully simple idea dressed up in a lot of careful engineering: save one execution context, load another, and repeat fast enough that it looks simultaneous. Threads are created via syscalls like clone() that control exactly which resources get shared versus duplicated. Synchronization between threads relies on atomic instructions (LOCK-prefixed operations on x86, load/store-exclusive pairs on ARM) and memory barriers, since without them, threads sharing memory can observe stale or reordered data. Understanding this layer makes concurrency bugs in higher-level languages far less mysterious, because you can trace exactly where the guarantees (or lack of them) actually come from.

20. References

• Intel® 64 and IA-32 Architectures Software Developer’s Manual, Volume 3A: System Programming Guide (Chapter on Multiple-Processor Management)
• AMD64 Architecture Programmer’s Manual, Volume 2: System Programming
• ARM Architecture Reference Manual for A-profile architecture (Memory Model and Exclusive Access sections)
• Linux kernel source, clone(2) man page and arch/x86/entry, arch/arm64/kernel/entry.S
• GNU Binutils / as documentation