How does the operating system manage main memory for running processes

Every time you double-click an application icon, an enormous amount of coordination happens behind the scenes before you even see a splash screen. The operating system has to find space for that program in memory, load its code and data, keep it isolated from every other running program, and be ready to move things around the moment memory gets tight. Most of us never think about it, which is honestly a sign of how well it works.

I want to use this article to walk through, in detail, how an operating system actually manages main memory for the dozens (or hundreds) of processes running on a typical system at any given moment. I’ll cover the core concepts, the data structures involved, the algorithms used for allocation and replacement, and how this plays out differently across Linux, Windows, Android, iOS, and traditional UNIX.

The Core Problem

Main memory (RAM) is a shared, finite resource. At any moment, a modern operating system might be juggling browser processes, background services, a music player, an IDE, and dozens of kernel threads, all of which need memory. The OS has to solve several problems simultaneously:

• How much memory should each process get?
• Where in physical memory should that memory live?
• How do we keep one process from reading or corrupting another process’s memory?
• What happens when total memory demand exceeds physical capacity?
• How do we make all of this fast enough that users don’t notice the overhead?

Process Address Spaces

Every process is given its own virtual address space, an abstraction that makes the process believe it has access to a large, contiguous block of memory, starting typically at address zero and extending up to some architecture-defined limit (commonly 4GB for 32-bit systems, and an enormous range like 128TB or more for 64-bit systems, though only a fraction is actually used).

This virtual address space is divided into regions:

High Addresses
+--------------------+
|      Stack         |  <- grows downward
|         |          |
|         v          |
|                    |
|         ^          |
|         |          |
|      Heap          |  <- grows upward
+--------------------+
|  BSS (uninitialized)|
+--------------------+
|  Data (initialized)|
+--------------------+
|      Text (code)   |
+--------------------+
Low Addresses

The critical insight is that this virtual layout is entirely independent of where the process’s data actually sits in physical RAM. The OS, with hardware assistance, translates virtual addresses to physical addresses on the fly.

Address Translation and the MMU

The Memory Management Unit (MMU), a piece of hardware built into the CPU, handles the translation between virtual and physical addresses. The OS maintains page tables that describe this mapping for each process. When a process accesses a memory address, the MMU consults the page table (often accelerated via a cache called the Translation Lookaside Buffer, or TLB) to determine the corresponding physical address.

This translation layer is what makes process isolation possible: two processes can both believe they own memory address 0x00400000, but the MMU maps each process’s virtual 0x00400000 to a completely different physical location, so they never actually collide.

Paging: The Dominant Memory Management Scheme

Virtually every modern general-purpose OS manages physical memory using paging. Physical memory is divided into fixed-size frames (commonly 4KB, though huge pages of 2MB or 1GB are used for performance-sensitive workloads). A process’s virtual address space is divided into pages of the same size. The OS maintains a mapping, the page table, that assigns physical frames to virtual pages as needed.

Key benefits of paging:

• Physical memory doesn’t need to be contiguous for a process’s allocation, virtually eliminating external fragmentation.
• Processes can be allocated exactly the number of pages they need (subject to internal fragmentation within the last page).
• Pages not currently in physical memory can be marked “not present” and loaded on demand (demand paging), or swapped out under memory pressure.

Demand Paging

Rather than loading an entire program into memory at launch, modern operating systems use demand paging: pages are loaded into RAM only when actually accessed. When a process references a page that isn’t currently in memory, a page fault occurs. The OS catches this fault, locates the required page (typically from the executable file on disk, or from swap space if it was previously swapped out), loads it into a free frame, updates the page table, and resumes the process, all typically transparent to the running program.

This lazy-loading approach dramatically speeds up process startup and reduces wasted memory for code paths that are never actually executed during a given run.

Page Replacement: What Happens When Memory Is Full

When physical memory fills up and a page fault requires a new frame, the OS must choose an existing page to evict. This is called page replacement, and the algorithm used has a major impact on system performance.

Common page replacement algorithms include:

• FIFO (First-In, First-Out): Evicts the oldest loaded page. Simple but can perform poorly (Belady’s anomaly can even cause more page faults with more frames under FIFO).
• LRU (Least Recently Used): Evicts the page that hasn’t been accessed for the longest time, based on the reasonable assumption that recently used pages are likely to be used again soon. True LRU is expensive to implement precisely, so most real systems use approximations.
• Clock algorithm (Second-Chance): A practical approximation of LRU, using a reference bit and a circular scan through frames, giving pages a “second chance” if they were recently accessed. This is what Linux’s page reclaim logic is conceptually based on, refined into more sophisticated multi-list approaches.
• LFU (Least Frequently Used): Evicts the page accessed least often overall, useful in some workloads but can wrongly penalize pages that were heavily used long ago but not recently.

Linux, specifically, uses an approach involving active and inactive lists of pages, moving pages between them based on access patterns, effectively implementing an approximated LRU with additional heuristics for file-backed versus anonymous memory.

Memory Allocation for Processes: A Step-by-Step View

Here is roughly what happens when a new process is created and starts consuming memory, using Linux terminology as an example, though the general flow is similar across platforms:

1. Process creation (fork/exec on Linux, CreateProcess on Windows): The OS creates a new process control block and a fresh virtual address space, largely empty at first, with page table entries mostly unmapped.
2. Loading the executable: The OS memory-maps the executable file’s segments (code, initialized data) into the new virtual address space, without necessarily loading their actual contents into RAM yet.
3. First access triggers page faults: As the process begins executing instructions, page faults occur for each page not yet resident, and the OS loads them from disk on demand.
4. Dynamic memory requests: As the process calls malloc() or similar, the C library requests more virtual memory from the OS (via brk/sbrk or mmap on Linux, VirtualAlloc on Windows), which the OS grants by extending the process’s virtual address space, again typically without immediately committing physical frames.
5. Memory pressure management: If system-wide memory becomes scarce, the OS’s page reclaim/replacement logic starts evicting less-used pages, potentially swapping them to disk, to make room.
6. Process termination: When the process exits, the OS reclaims all its physical frames and destroys its page tables, making that memory available for other processes.

Process Isolation and Protection

A critical responsibility of main memory management is protection: ensuring one process cannot read or write another process’s memory without explicit permission (such as through shared memory mechanisms). This is enforced through:

• Separate page tables per process, so no process can construct a valid mapping to another process’s physical frames without OS cooperation.
• Privilege levels (kernel mode vs user mode), where hardware enforces that user-mode code cannot directly manipulate page tables or access kernel memory.
• Access permission bits on pages (read, write, execute), enforced by the MMU, which is why attempting to execute code in a non-executable data page typically triggers a fault (a protection central to defenses like DEP/NX in Windows and similar mechanisms elsewhere).

Shared Memory and Copy-on-Write

Not all memory management is about isolation; sometimes processes need to share memory deliberately, for performance or communication purposes.

• Shared libraries: Common libraries (like libc) are mapped into multiple processes’ address spaces but backed by the same physical frames, saving substantial memory across a system running many processes that use the same library.
• Copy-on-Write (COW): When a process forks, rather than immediately duplicating all its memory, the OS maps the child’s pages to the same physical frames as the parent, marking them read-only. Only when either process attempts to write to a shared page does the OS actually copy that specific page, a huge efficiency win for the common case where a forked child immediately calls exec and discards most inherited memory anyway.

Platform-Specific Notes

Linux

Linux’s memory management is highly sophisticated, combining demand paging, the buddy allocator for physical frames, slab/slub allocators for kernel objects, transparent huge pages for performance, and a tunable reclaim subsystem (kswapd, memory cgroups for containerized workloads). Tools like /proc/meminfo, top, and vmstat expose this behavior directly.

Windows

Windows uses a broadly similar model: a Virtual Memory Manager handles paging, working sets (the set of pages a process currently has resident), and a paging file for overflow. Windows also introduced concepts like Address Windowing Extensions historically, and more recently, memory compression (introduced in Windows 10) as an intermediate step before actual disk-based paging.

Android

Android layers its own process lifecycle management on top of the Linux kernel’s memory management, using the Low Memory Killer/lmkd to terminate background apps under memory pressure rather than relying purely on swapping, given flash storage constraints.

iOS

iOS uses a broadly similar strategy: compressed memory and aggressive background app termination rather than disk-based swapping, prioritizing storage longevity and consistent performance on constrained mobile hardware.

UNIX (Solaris, BSD)

Traditional UNIX systems pioneered many of these ideas. Solaris’s virtual memory system, for instance, directly influenced page-cache unification concepts that Linux later adopted, treating file-backed pages and process memory pages under a unified reclaim framework.

Best Practices for Developers and Administrators

• Understand your application’s working set size, and size physical RAM (and container memory limits, if applicable) accordingly.
• Use memory-mapped files (mmap) for large data sets where appropriate, letting the OS’s paging system handle loading efficiently.
• Monitor page fault rates, not just raw memory usage, since a high major page fault rate (faults requiring disk I/O) is a stronger indicator of memory pressure than memory usage percentage alone.
• In containerized environments, be aware that cgroup memory limits interact directly with the kernel’s reclaim behavior, and misconfigured limits can trigger unnecessary reclaim or OOM kills.

Summary

Operating systems manage main memory for running processes through a layered system built on virtual memory, paging, demand loading, and careful page replacement policies, all enforced with hardware assistance from the MMU. This lets systems run far more processes than would fit in physical RAM alone, while maintaining strong isolation between them. The fundamental theory is shared across Linux, Windows, Android, iOS, and UNIX, though each platform tunes the details based on its own hardware constraints and performance priorities.

Frequently Asked Questions

What is the difference between virtual memory and physical memory? Physical memory is the actual RAM installed in a machine. Virtual memory is an abstraction the OS provides to each process, making it appear as though it has access to a large, private, contiguous address space, regardless of how physical memory is actually organized or shared.

Why do processes not share memory by default? Isolation prevents bugs or malicious code in one process from corrupting or reading another process’s data, which is essential for both stability and security.

What causes a page fault, and is it always bad? A page fault occurs when a process accesses a virtual address not currently mapped to physical memory. Minor page faults (where the page just needs a fresh frame, like on first touch of newly allocated memory) are normal and cheap. Major page faults (requiring an actual disk read) are more costly and, if frequent, indicate memory pressure.

How does the OS decide how much memory to give each process? There isn’t a fixed quota by default; the OS grants memory as processes request it, up to system-wide or configured limits, and reclaims pages under memory pressure based on usage patterns.

References

• Silberschatz, Galvin, and Gagne, Operating System Concepts, Wiley.
• The Linux Kernel Documentation on memory management: kernel.org
• Microsoft Learn documentation on the Windows Virtual Memory Manager
• Android Open Source Project documentation on the Low Memory Killer