COEN 180

Memory Hierarchy

Memory Hierarchy

We are therefore forced to recognize the possibility of constructing a hierarchy of memories, each of which has greater capacity than the preceding but which is less quickly accessible.

A. W. Burks, H. H. Goldstine, J. von Neumann: Preliminary Discussion of the Logical Design of Electronic Computing Instrument, Part I, Vol. I, Report prepared for the U.S. Army Ord. Dept. 28 June 1946

Computer Components

There are three basic hardware modules (Bell, Newell: Computer Structures, 1971):

Processors Memory Communication

Memory / Storage Evaluation

Costs Capacity Speed Reliability Volatility Other criteria such as

transportability

Memory / Storage Evaluation

Memory Hierarchy

Exploiting the Memory Hierarchy

Not all stored data is equally important.

Put important data in the upper ranges of the memory / storage hierarchy.

Put unimportant data in the lower ranges.

Exploiting the Memory Hierarchy

Locality Spatial Locality:

Data is more likely to be accessed if neighboring data is accessed.

Temporal Locality: Data is more likely to be accessed if it

has been recently accessed.

Exploiting the Memory Hierarchy Executables

Program executions tend to spend a great portion of time in loops.

Spatial locality: if a statement in the loop is executed, then so are the statements surrounding it.

Temporal locality: if a statement is executed, it is likely to be executed again.

Exploiting the Memory Hierarchy Relational Databases

Store data in relations Relation consists of fields Often with Record ID. Stored in a B+ tree or in a (linear) hash table.

Spatial Locality Accessing all records in order, records are stored

in B+ tree. Makes sense to move records in bunches from

disk / tape to main memory. Typical transaction has no spatial locality.

Accesses a record here and there all over the place. No spatial locality.

Exploiting the Memory Hierarchy Relational Databases

Temporal Locality Some records are hot, most are cold.

Records of current students vs. records of graduates.

Active accounts in a bank database. Current patients versus other patients.

Some transactions look at the same record several times (due to inefficiencies).

Exploiting the Memory Hierarchy

File System Temporal Locality:

Few files are frequently accessed (OS kernel, killer apps, data in current projects).

Most are written and never read again. Spatial Locality:

Not only individual files, but also directories can become hot.

Exploiting the Memory Hierarchy

Caching strategy: Keep popular items in expensive,

small, and fast memory. Keep less popular items in cheap, big,

and slow memory. Use spatial & temporal locality to

guess what items are popular.

Exploiting the Memory Hierarchy

Use Caches throughout the Memory Hierarchy Registers & SRAM versus DRAM (main

memory) Disk buffer versus Disk platter Disks versus Tapes Local Storage versus Remote Storage

Cache Analysis Assume two levels of memory:

Cache: fast, small, expensive. Main: slow, large, cheap.

Data access: Read Write Assume read only access Given data identifier, read data.

Cache Analysis

Hit rate h: chances of satisfying read request from cache.

Miss rate m: chances having to read from main.

h + m = 1

Cache Analysis

tC Time to read from cache tM Time to read from main tC < tM

Expected access time = system access time tSystem = h tC + m tM

Hit rates high: system access time close to cache access time

Caching Issues Larger cache slower access times Selection of items into cache

Good selection takes more time and uses more resources

Cache architecture depends on the situation Cache between main memory and processor Cache between disk drive(s) and main memory Cache between remote and local storage