Memory Systems

The DRAM subsystem has a deep hierarchy:

• Channel: Independent memory controller connection with its own data bus (e.g., 64-bit wide). Multiple channels multiply bandwidth.
• DIMM (Dual Inline Memory Module): Physical stick of memory containing multiple chips. Multiple DIMMs can share a channel.
• Rank: A group of DRAM chips that operate in lockstep on the same DIMM. A 64-bit channel uses 8 chips per rank (each chip contributes 8 bits).
• Chip: Individual DRAM chip containing multiple banks.
• Bank: Independent array within a chip with its own row buffer (sense amplifiers). Banks can be accessed concurrently.
• Row/Column: Within a bank, data is organized as a 2D array. A row is activated into the row buffer; columns are read from the row buffer.

Key hierarchy: Channel > DIMM > Rank > Chip > Bank > Row/Column

Transferring a 64-byte cache block: Across a rank, each of 8 chips contributes 8 bytes. This requires 8 column reads (each 8B burst) to fill a 64B cache line.

DRAM banks use a row buffer (implemented by sense amplifiers) to hold an entire activated row.

Three DRAM commands:

1. ACTIVATE (ACT / RAS): Opens a row — copies the entire row from the DRAM array into the row buffer. Destructive to the array (must write back eventually). Takes t_RAS time.
2. READ/WRITE (CAS): Accesses a specific column within the open row buffer. Much faster than a full row activation. Takes t_CAS time.
3. PRECHARGE (PRE): Closes the current row — writes the row buffer contents back to the array and prepares the bank for a new row activation. Takes t_RP time.

Row buffer states:

• Row buffer hit: Requested row is already open in the row buffer → just CAS → fastest (t_CAS)
• Row buffer miss (closed bank): No row is open → ACT + CAS (t_RAS + t_CAS)
• Row buffer conflict: A different row is open → PRE + ACT + CAS → slowest (t_RP + t_RAS + t_CAS)

DRAM capacitors leak charge and must be refreshed periodically (typically every 64 ms for the entire memory).

Refresh strategies:

• Burst refresh: Refresh all rows back-to-back. Memory is unavailable for a long burst. Simpler but causes long stalls.
• Distributed refresh: Spread refresh commands throughout the 64ms window. Each refresh command handles a few rows. Shorter individual stalls but more frequent interruptions.

Impact: During refresh, the bank being refreshed cannot service memory requests → performance and energy overhead. As DRAM capacity grows, more rows need refreshing → refresh overhead increases.

Refresh penalty grows with capacity: More rows → more refresh commands → more time spent refreshing → less time available for actual accesses.

Multiple banks within a chip/rank enable bank-level parallelism:

• Each bank has its own row buffer and operates independently
• While one bank is activating a row, another can be servicing a CAS
• Interleaving accesses across banks hides latency

Multiple channels provide additional bandwidth:

• Each channel has its own data bus, command bus, and memory controller port
• Channels operate completely independently
• Two channels → 2x peak bandwidth

Address mapping/interleaving: How physical addresses are mapped to channels, ranks, banks, rows, and columns significantly affects performance.

• Using lower address bits for bank/channel selection provides better interleaving because lower bits have more entropy (vary more across consecutive accesses)
• Row interleaving: consecutive cache blocks go to different banks → maximizes bank-level parallelism for sequential access patterns

The memory controller decides the order in which queued memory requests are serviced. This order significantly impacts performance.

FCFS (First-Come, First-Served):

• Service requests in arrival order
• Simple, fair, but ignores row buffer state
• May cause unnecessary row conflicts when a row hit request is waiting behind a row conflict request

FR-FCFS (First-Ready, First-Come, First-Served):

• Priority 1: Row buffer hits first (they're fastest — CAS only)
• Priority 2: Among equal-priority requests, oldest first (FCFS)
• Maximizes row buffer hit rate → maximizes DRAM throughput
• But can be unfair — streaming accesses (many hits to same row) can starve other requests

Key insight: FR-FCFS prioritizes throughput over fairness. This can cause quality-of-service issues in multi-core/multi-application systems.

The total memory access latency has several components:

1. CPU to memory controller: On-chip interconnect latency
2. Controller latency / queuing: Time spent waiting in the memory controller queue + scheduling decision time
3. Controller to DRAM: Bus transfer of command (address/command bus)
4. Bank access latency:
   • Row hit: t_CAS (Column Access Strobe)
   • Row miss (closed): t_RAS + t_CAS
   • Row conflict: t_RP + t_RAS + t_CAS (Precharge + Activate + CAS)
5. DRAM to controller (data bus): Data transfer across the memory bus. For a 64B cache line at 64-bit bus width: 8 transfers (each 8 bytes)
6. Controller to CPU: On-chip interconnect back to the processor

Total latency can easily be 100+ ns for a row conflict, vs. ~50 ns for a row buffer hit.