The 5-Tier Memory Hierarchy Pyramid & Cache Memory Principles

In modern computer design, no single memory technology is simultaneously Ultra-Fast, Massive in Capacity, and Inexpensive. • The Trade-off: Faster memories (SRAM, Registers) require complex multi-transistor latch circuits costing more per bit and consuming more silicon area. Slower memories (DRAM, Flash SSD, Magnetic HDD) are dense and cheap per gigabyte but take hundreds of clock cycles to respond. • The Solution: A multi-level hierarchical storage system where frequently referenced data is cached close to the CPU, while massive inactive archives reside on secondary/tertiary storage.

Cache Memory is a small, ultra-fast semiconductor memory placed between the CPU core and Main RAM to bridge the processor-memory speed bottleneck. • Principle of Locality of Reference: 1. Temporal Locality (Locality in Time): If a memory location is referenced once, it will likely be referenced again soon (e.g., loop variables, function calls, counter indices). 2. Spatial Locality (Locality in Space): If a memory location is referenced, nearby contiguous addresses will likely be referenced soon (e.g., sequential array elements, adjacent program instructions). • Cache Hit vs Cache Miss: • Cache Hit: CPU finds the requested word directly in Cache. High speed, zero DRAM latency. • Cache Miss: CPU does not find word in Cache; it must fetch the entire block from slower Main Memory and update Cache. • Hit Ratio (H): Fraction of total memory accesses fulfilled by Cache: H = Hits / (Hits + Misses). • Average Memory Access Time (T_avg): T_avg = H * T_cache + (1 - H) * T_main.

1. Direct Mapping: Each block of main memory maps to exactly one specific line in Cache using line index bits (Formula: Cache Line = Main Memory Block Modulo Number of Cache Lines). Fast lookup, but high contention collision misses. 2. Fully Associative Mapping: Any main memory block can be loaded into ANY available line in Cache. Zero conflict misses, but requires expensive parallel comparator circuits. 3. Set Associative Mapping (Modern Industry Standard): Compromise between Direct and Associative. Cache is divided into sets of N lines (e.g., 2-way, 4-way, 8-way Set Associative). Block maps to a specific set, but can occupy any line within that set.