TLDR: Why and How Cache Locality Can Make Your Code Faster

Date: 2025-09-04 Source: https://arpitbhayani.me/blogs/cpu-cache-locality

Overview

While we write code thinking about algorithms and data structures, the CPU is quietly making millions of decisions about what data to keep close and what to evict from its precious cache memory. One of the most significant factors determining whether your code runs fast or crawls is something that many engineers never directly interact with: CPU cache locality.

Key Points

• One of the most significant factors determining whether your code runs fast or crawls is something that many engineers never directly interact with: CPU cache locality.
• Why Cache Exists: Speed v/s capacity is the fundamental trade-off here.
• Types of Cache Locality: Temporal Locality Temporal locality means that recently accessed data is likely to be accessed again soon.
• Measuring Cache Performance: CPUs provide hardware counters that let you measure cache behavior directly using perf.