Cache Hierarchy Design-Space Exploration

The problem

Cache design questions are usually posed as miss-rate questions. Miss rate is a poor objective on its own. A larger, more associative cache lowers misses while raising access energy, leakage, and area, and the access latency it adds can erase the benefit it bought. Answering “is this configuration better” requires holding all of those in view at once.

The two halves of that answer come from different tools. Miss behavior comes from simulating a memory reference stream. Energy, leakage, and area come from a circuit-level model of the array. This project connects them.

Approach

The functional half is a trace-driven simulator. It consumes memory reference traces and models a configurable hierarchy: cache size, block size, associativity, replacement policy, write policy, and inclusion behavior across levels. Trace-driven is the right choice here because cache behavior depends on the reference stream, not on the timing of the pipeline that produced it. It costs accuracy on anything timing-dependent, which is a real limitation I accepted deliberately.

The physical half is HP CACTI. For each configuration in the sweep, CACTI supplies per-access dynamic energy, leakage power, access latency, and area for the given technology node. The driver generates the configuration space, runs both tools, and joins the outputs into a single frame in pandas.

From the joined data I compute the metrics that combine the two halves. AMAT folds hit time and miss penalty against the simulated miss rate. Energy-delay product weights an energy saving against the runtime cost of achieving it, which is what stops the search from collapsing onto trivially small low-power caches that are slow. Leakage is tracked separately from dynamic energy because it scales with area and idle time rather than with access count, and lumping the two hides that.

Because the objectives conflict, I do not report a single winner. The driver computes Pareto fronts over the chosen objective pairs and identifies the non-dominated configurations. Everything below the front can be discarded honestly; choosing among points on the front is a design decision that depends on the target, not something the tooling should pretend to settle.

I validate the simulator against pycachesim on shared configurations. Two independent implementations agreeing does not prove either is right, but a disagreement always means one of them has a bug, and it caught mistakes in my replacement and write-allocate handling. Plot generation is automated as part of the sweep so figures regenerate with the data rather than drifting from it.

Limitations

A trace-driven model has no notion of overlap. It cannot represent memory-level parallelism, so it systematically overstates the penalty of misses that a real out-of-order core would hide. Results are only as representative as the traces, and the conclusions are workload-specific by construction. CACTI’s numbers are analytical estimates for a technology node, useful for comparing configurations against each other and not to be read as absolute values for real silicon.