How Does Caching Affect Performance in Embedded Systems

I remember benchmarking an image processing routine on a Cortex-A-based embedded Linux board and getting wildly inconsistent timing results — sometimes fast, sometimes mysteriously slow, with no code changes between runs. The culprit turned out to be cache behavior: how my data was laid out in memory was determining whether the CPU was hitting a fast cache or stalling on a slow external RAM access. That experience is what pushed me to actually understand caching at a level deep enough to design around it, rather than treating it as an invisible hardware detail. In this article I’ll explain how caching works in embedded systems, where it helps, where it hurts (especially in real-time contexts), and how to work with it deliberately.

What a Cache Actually Is

A cache is a small, extremely fast memory sitting between the CPU core and slower main memory (flash or external RAM), designed to exploit the fact that programs tend to access the same or nearby memory locations repeatedly (temporal and spatial locality). When the CPU requests data, the cache is checked first; if the data is there (a cache hit), it’s returned almost instantly. If not (a cache miss), the CPU has to wait for a much slower fetch from main memory, and the cache typically loads that data (and its neighbors) in for next time.

graph TD
    A[CPU Core] --> B{Data in Cache?}
    B -->|Cache Hit - fast| C[Return data in 1-few cycles]
    B -->|Cache Miss - slow| D[Fetch from Main Memory/Flash]
    D --> E[Load into Cache]
    E --> C

Why This Matters More in Embedded Than Desktop Computing

On a desktop CPU, caching is largely a performance-only concern — a cache miss makes things slower but not incorrect. In embedded systems, caching intersects with two things desktop programmers rarely worry about: hard real-time determinism and memory-mapped hardware registers, and getting either wrong isn’t just slow — it can be a functional bug.

The Determinism Problem

A cache miss can take 10-50x longer than a cache hit. If a hard real-time task’s execution time depends on whether its instructions and data happen to be cached, its Worst-Case Execution Time (WCET) becomes very difficult to bound — which, as I covered in the hard vs. soft real-time article, is unacceptable for safety-critical control loops.

gantt
    title Cache Hit vs Cache Miss Timing Variability
    dateFormat X
    axisFormat %L cycles
    section Best case - all cache hits
    Instruction fetch + execute :done, a1, 0, 20
    section Worst case - cache misses
    Instruction fetch (miss, wait for RAM) :crit, a2, 0, 150
    Execute :a3, 150, 20

This is exactly why many hard real-time embedded systems either disable caching for critical code paths, use cache-locking (pinning specific code/data permanently in cache so it can never miss), or avoid caches altogether by running critical routines from fast, deterministic tightly-coupled memory (TCM) instead.

The Coherency Problem with Memory-Mapped Peripherals

Peripheral registers (like a GPIO or UART data register) are mapped into the same address space as regular memory, but they represent live hardware state, not data to be cached. If the CPU’s data cache caches a read of a peripheral register, subsequent reads might return a stale cached value instead of the register’s actual current state — silently breaking functionality.

/* DANGEROUS if this memory region is cacheable:
   the CPU might return a stale cached value instead of
   reading the peripheral's live status register */
#define UART_STATUS_REG (*(volatile uint32_t *)0x40011000)

/* Correct approach: mark peripheral memory regions as
   "Device" or "Strongly Ordered" (non-cacheable) in the
   MPU/MMU configuration, AND use `volatile` in firmware
   so the compiler doesn't optimize away repeated reads */
while (!(UART_STATUS_REG & UART_TXE_FLAG)) {
    /* wait for transmit-empty flag - must read the real register every time */
}

On Cortex-M/A systems with an MPU or MMU, I explicitly configure peripheral address ranges as non-cacheable, non-bufferable memory regions, separate from cacheable SRAM/flash regions used for regular code and data.

/* Example: ARM Cortex-M7 MPU region configuration marking a
   peripheral address range as Device memory (non-cacheable) */
MPU_Region_InitTypeDef MPU_InitStruct = {0};

MPU_InitStruct.Enable = MPU_REGION_ENABLE;
MPU_InitStruct.BaseAddress = 0x40000000; /* peripheral base */
MPU_InitStruct.Size = MPU_REGION_SIZE_512MB;
MPU_InitStruct.AccessPermission = MPU_REGION_FULL_ACCESS;
MPU_InitStruct.IsShareable = MPU_ACCESS_SHAREABLE;
MPU_InitStruct.IsCacheable = MPU_ACCESS_NOT_CACHEABLE; /* critical */
MPU_InitStruct.IsBufferable = MPU_ACCESS_NOT_BUFFERABLE;
HAL_MPU_ConfigRegion(&MPU_InitStruct);

Where Caching Genuinely Helps Embedded Performance

On higher-end embedded processors (Cortex-A series running Linux, or Cortex-M7 with tightly-coupled and cached memory), caching provides very real performance benefits for non-time-critical, compute-heavy workloads:

• Image/signal processing — iterating over large buffers benefits enormously from spatial locality
• Interpreted/scripted logic (Python on embedded Linux, Lua scripting engines) — repeated bytecode execution hits cache heavily
• File system and networking stacks — repeated access to the same data structures

Writing Cache-Friendly Embedded C Code

I structure data access patterns deliberately to maximize cache hits:

/* Cache-unfriendly: column-major access on a row-major array
   jumps across memory, causing a cache miss almost every iteration */
for (int col = 0; col < WIDTH; col++) {
    for (int row = 0; row < HEIGHT; row++) {
        process(image[row][col]);
    }
}

/* Cache-friendly: row-major access matches memory layout,
   so consecutive accesses hit the same cache line */
for (int row = 0; row < HEIGHT; row++) {
    for (int col = 0; col < WIDTH; col++) {
        process(image[row][col]);
    }
}

graph LR
    A[Row-major array in memory] --> B[Row-major access pattern]
    B --> C[Sequential addresses - high cache hit rate]
    A --> D[Column-major access pattern]
    D --> E[Scattered addresses - cache miss on nearly every access]

Cache Levels and Typical Embedded Configurations

Many mid-range Cortex-M parts don’t have a true cache at all but instead use flash prefetch buffers and wait-state configuration to hide flash access latency — a related but simpler mechanism that still needs correct configuration (like enabling the ART Accelerator on STM32 parts) to get advertised clock-speed performance.

/* STM32 example: enabling flash prefetch and instruction cache
   for performance at high clock speeds */
void Flash_Performance_Config(void)
{
    __HAL_FLASH_PREFETCH_BUFFER_ENABLE();
    __HAL_FLASH_INSTRUCTION_CACHE_ENABLE();
    __HAL_FLASH_DATA_CACHE_ENABLE();
}

Cache Locking for Real-Time Determinism

Some processors let critical routines be locked permanently into cache, guaranteeing they’ll always execute at cache-hit speed regardless of what else the system is doing — giving predictable timing without sacrificing all caching benefits elsewhere.

graph TD
    A[Cache] --> B[Locked Region: Critical ISR code - always resident]
    A --> C[Normal Region: General application code - evictable]
    D[Cache Miss on normal region] -->|evicts| C
    D -.->|never evicts| B

Benchmarking Cache Impact in Practice

I never take claims about cache performance on faith — I measure. A typical benchmarking approach on a Cortex-A or Cortex-M7 platform:

#include "core_cm7.h"

uint32_t Measure_Cycles(void (*func)(void))
{
    DWT->CYCCNT = 0;
    DWT->CTRL |= DWT_CTRL_CYCCNTENA_Msk;

    func();

    return DWT->CYCCNT;
}

void Benchmark_Cache_Impact(void)
{
    uint32_t cycles_cache_disabled, cycles_cache_enabled;

    SCB_DisableDCache();
    cycles_cache_disabled = Measure_Cycles(process_image_buffer);

    SCB_EnableDCache();
    /* run once to warm the cache before measuring steady-state performance */
    process_image_buffer();
    cycles_cache_enabled = Measure_Cycles(process_image_buffer);

    printf("Without cache: %lu cycles\n", cycles_cache_disabled);
    printf("With cache:    %lu cycles\n", cycles_cache_enabled);
}

On real workloads I’ve profiled this way, cache-friendly image and buffer processing routines commonly run 3-8x faster with the data cache enabled and properly warmed, which is a large enough difference that it genuinely changes architectural decisions — like whether a given algorithm can hit a required frame rate at all.

Cache-Aware Data Structure Design

Beyond simple loop ordering, I design data structures themselves with cache line size in mind (typically 32 or 64 bytes on embedded Cortex-A/M7 cores). Structuring data so that fields accessed together in a hot loop are physically adjacent in memory (an approach sometimes called “structure of arrays” versus “array of structures,” depending on the access pattern) can dramatically reduce cache misses.

/* Array-of-structures: each iteration pulls in an entire
   struct per cache line, wasting bandwidth if only .x is needed */
typedef struct { float x, y, z, temperature, pressure; } sensor_reading_t;
sensor_reading_t readings[1000];

float sum_x_aos(void) {
    float sum = 0;
    for (int i = 0; i < 1000; i++) sum += readings[i].x; /* wastes cache line on unused fields */
    return sum;
}

/* Structure-of-arrays: all .x values are contiguous, so each
   cache line loaded is fully utilized for this access pattern */
float x_values[1000], y_values[1000], z_values[1000];

float sum_x_soa(void) {
    float sum = 0;
    for (int i = 0; i < 1000; i++) sum += x_values[i]; /* every loaded byte is used */
    return sum;
}

Which layout is actually better depends entirely on the access pattern — if code typically needs all fields of one reading together, array-of-structures is more cache-friendly; if code typically scans one field across many readings (as in the example above), structure-of-arrays wins. I choose based on profiling the dominant access pattern in the actual application, not by assuming one layout is universally superior.

How Compiler Optimization Interacts with Cache Behavior

Compiler optimization levels change generated code in ways that directly interact with caching, and I’ve learned to treat this as a deliberate design consideration rather than an afterthought. Higher optimization levels (-O2/-O3) tend to inline functions and unroll loops, which increases instruction cache footprint — sometimes enough that a routine that fit comfortably in I-cache at -O1 starts causing cache misses at -O3, paradoxically making “more optimized” code slower in practice. Link-Time Optimization (LTO) and Profile-Guided Optimization (PGO) can help by making smarter inlining decisions based on actual hot-path data rather than local heuristics.

/* Explicitly marking a function to be placed in a specific
   memory section, useful for keeping hot ISR code in
   fast, deterministic memory (like TCM) regardless of what
   the compiler's general code placement would otherwise choose */
__attribute__((section(".itcm_text")))
void Critical_ISR_Handler(void)
{
    /* time-critical code guaranteed to execute from TCM,
       bypassing any dependency on cache hit/miss behavior */
}

For genuinely cache- and timing-sensitive routines, I don’t leave placement entirely to compiler defaults — I use linker section attributes to explicitly place critical functions in TCM or a locked cache region, and I always re-benchmark after any optimization level or compiler version change, since cache interaction effects can be surprisingly non-intuitive and don’t always move performance in the expected direction.

Prefetching and Branch Prediction Alongside Caching

Caching doesn’t work alone — it’s typically paired with prefetching and, on higher-end embedded cores, branch prediction, both of which affect real-world performance in similar ways. A prefetcher speculatively loads data or instructions into cache before they’re explicitly requested, based on observed access patterns (like sequential memory access), which is why the row-major loop example earlier in this article benefits doubly: it’s both cache-friendly and prefetcher-friendly, since the prefetcher can accurately predict the next several addresses that will be needed. Branch mispredictions on cores with deep pipelines (more common on Cortex-A than Cortex-M) cause a pipeline flush that has a similar unpredictability problem to a cache miss — which is another reason hard real-time code on such cores favors simple, predictable control flow (avoiding deeply nested conditionals or data-dependent branching in the most timing-critical paths) over cleverness that might average out well but occasionally spikes badly.

Real-World Applications

• Embedded Linux media players/gateways (Cortex-A) — cache-conscious buffer handling directly affects video decode throughput
• Automotive infotainment vs. safety domains — infotainment SoCs use aggressive caching for UI performance, while the separate safety-critical MCU domain (running ABS/airbag logic) deliberately avoids or tightly controls caching for determinism
• Cortex-M7 motor control — critical control-loop code placed in TCM for guaranteed zero-wait-state execution, while less critical logic runs from cached flash
• Industrial vision systems — image processing pipelines optimized for cache-friendly access patterns to hit real-time frame rate targets

Debugging Cache-Related Bugs

A classic embedded bug signature: code that works perfectly with the debugger’s cache-disabling behavior or at -O0 optimization, but misbehaves at -O2/-O3 with caching enabled. My checklist when I suspect a cache issue:

1. Check whether peripheral/DMA-shared memory regions are correctly marked non-cacheable
2. Verify volatile is used on any variable modified by hardware/DMA/ISR outside the compiler’s visibility
3. For DMA buffers on cached cores, explicitly invalidate/clean the cache before/after DMA transfers so the CPU and DMA controller see consistent data
4. Profile with and without cache enabled to isolate whether a “random” bug is actually a caching/coherency issue

/* Cache maintenance around a DMA transfer on a Cortex-M7 style core */
void Prepare_DMA_Buffer_For_Transfer(uint8_t *buf, uint32_t size)
{
    SCB_CleanDCache_by_Addr((uint32_t *)buf, size); /* flush CPU writes to RAM before DMA reads */
}

void Handle_DMA_Complete(uint8_t *buf, uint32_t size)
{
    SCB_InvalidateDCache_by_Addr((uint32_t *)buf, size); /* discard stale cache before CPU reads new DMA data */
}

Frequently Asked Questions

Do all microcontrollers have a cache? No — many low-to-mid-range MCUs (Cortex-M0/M0+/M3, most AVR parts) have no true data/instruction cache, relying instead on flash prefetch buffers or simply running at speeds where flash latency isn’t a major bottleneck. Caches become common on higher-performance Cortex-M7 and Cortex-A parts.

Why would I ever disable caching in an embedded system? For hard real-time code paths where deterministic, bounded execution time matters more than average-case speed, and for any memory region that maps to live hardware (peripheral registers) rather than true data.

What’s the difference between a cache and tightly-coupled memory (TCM)? A cache automatically and transparently holds a subset of main memory contents with hit/miss behavior that’s hard to predict exactly. TCM is a small dedicated memory block the CPU accesses directly with guaranteed zero-wait-state timing — code explicitly placed there always runs at full speed, with no “miss” possible.

Can caching cause actual bugs, not just performance issues? Yes — most commonly through stale reads of memory-mapped peripheral registers, or through CPU/DMA cache coherency issues where the CPU cache and DMA-written RAM disagree on the current contents of a buffer. Both require explicit cache configuration or maintenance operations to avoid.

Summary

Caching in embedded systems is a double-edged tool: it can dramatically improve throughput for compute-heavy, non-critical workloads by exploiting locality of reference, but it introduces timing unpredictability that’s incompatible with hard real-time guarantees, and it can cause subtle correctness bugs around memory-mapped peripherals and DMA buffers if not configured deliberately. Understanding exactly which memory regions are cached, when cache maintenance operations are needed, and when to bypass caching entirely in favor of deterministic memory like TCM is what separates embedded code that merely runs from embedded code that’s actually reliable under real-world timing constraints.

References

• ARM Cortex-M7 Cache and TCM Technical Reference Manual
• ARM Cortex-A Series Programmer’s Guide – Caches
• STM32H7 ART Accelerator and Cache Application Note
• ARM Cache Maintenance Operations for Cortex-M7 (DMA Coherency)
• FreeRTOS and Cache Coherency Considerations