Feedback Directed Prefetching Santhosh Srinath Onur Mutlu Hyesoon Kim Yale N. Patt §¥ ¥ §

Feedback Directed PrefetchingSanthosh Srinath

Onur MutluHyesoon KimYale N. Patt

Problem

Prefetching can significantly improve performance When prefetches are accurate And timely

However, Prefetching can also significantly degrade performance Due to Memory Bandwidth impact Pollution of the cache

HPCA-13 Feedback Directed Prefetching 2

Feedback Directed Prefetching is a comprehensive mechanism which reduces the negative effects of prefetching as well as improves the positive effects

Solution

Feedback Directed Prefetching 3

Outline

Background and Motivation

Feedback Directed Prefetching (FDP) Metrics and How to collect How to adapt

Prefetcher Aggressiveness Cache Insertion Policy for Prefetches

Results

HPCA-13

Prefetch Distance

Prefetch Degree

Predicted StreamPredicted Stream

Background (Prefetcher Aggressiveness)

Access Stream

PmaxPrefetch Distance

PmaxVery Conservative

PmaxMiddle of the Road

PmaxVery Aggressive

Prefetch DegreeX+1

HPCA-13

Background (Prefetcher Aggressiveness) Very Aggressive

Well ahead of the load access stream Hides memory access latency better More speculative

Very Conservative Closer to the load access stream Might not hide memory access latency completely Reduces potential for cache pollution and

bandwidth contention

HPCA-13

No PrefetchingVery Conservative

Middle-of-the-RoadVery Aggressive

Motivation

Very Aggressive improves average performance by 84% However it can also significantly reduce performance on some benchmarks

48% 29%

HPCA-13

Outline

Feedback Directed Prefetching (FDP) Metrics and How to collect How to adapt

Results

HPCA-13 7Feedback Directed Prefetching

Feedback Directed Prefetching Comprehensive mechanism which takes in

account: Prefetcher Accuracy Prefetcher Lateness Prefetcher-caused Cache Pollution

Adapts Prefetcher Aggressiveness Cache Insertion Policy for Prefetches

HPCA-13

Metrics

Prefetch Accuracy

Prefetch Lateness

Prefetcher-caused Cache Pollution

HPCA-13

Prefetch Accuracy

Useful Prefetches are referenced by the demand requests when in L2

Memory Sent to Prefetches ofNumber

Prefetches UsefulofNumber Accuracy Prefetcher

HPCA-13

Prefetch Accuracy

Low Accuracy More likely that Prefetching can reduce performance

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

Prefetcher Accuracy

HPCA-13

Prefetch Accuracy

Implementation pref-bit added to each L2 tag-store entry Tracked using two counters: pref_total,

used_total

pref_total

used_totalAccuracy Prefetcher

HPCA-13

Prefetch Lateness

Measure of how timely prefetches are Used to determine if increasing the

aggressiveness helps Implementation

pref-bit added to each L2 MSHR entry New counter: late_total

Prefetches UsefulofNumber

Prefetches Late ofNumber LatenessPrefetch

used_total

late_total LatenessPrefetch

HPCA-13

Prefetcher-caused Cache Pollution

Measure of the disturbance caused by prefetched data in the cache

Used to determine if the prefetcher is evicting useful data from the cache

Misses Demand ofNumber

Prefetcher by the caused Misses Demand ofNumber

Pollution Cache causedPrefetcher

HPCA-13

Prefetcher-caused Cache Pollution (2)

Hardware Implementation Insight – this does not need to be exact Track pollution using Pollution filter

Based on Bloom Filter concept Bit set when a prefetch evicts a demand miss Bit reset when a prefetch is serviced

Two Counters – pollution_total, demand_total

aldemand_tot

totalpollution_Pollution Cache caused-Prefetcher

HPCA-13

Feedback Directed Prefetching Comprehensive mechanism which takes in

account: Prefetcher Accuracy Prefetcher Lateness Prefetcher-caused Cache Pollution

Adapts Prefetcher Aggressiveness Cache Insertion Policy

How to adapt? Prefetcher Aggressiveness Dynamic Configuration Counter

Current Aggressiveness

Distance Degree

1 Very Conservative 4 1

2 Conservative 8 1

3 Middle-of-the-Road 16 2

4 Aggressive 32 4

5 Very Aggressive 64 4

HPCA-13

Improve TimelinessReduce Cache Pollution

High Accuracy

Not-Late

Polluting

Decrease

Increase

How to adapt? Prefetcher Aggressiveness (2)

For Current Phase, based on static thresholds, classify Accuracy Lateness Cache-Pollution caused by Prefetches

Med Accuracy

Not-Poll

Increase

Polluting

Decrease

Low Accuracy

Not-Poll

Not-Late

No Change

Decrease

Reduce memory bandwidth usage and

Cache Pollution

HPCA-13

How to Adapt?Cache Insertion Policy for Prefetches Why adapt?

Reduce the potential for cache pollution Classify Cache Pollution based on static

thresholds: Low – Insert at MID(n/2) Position

Eg: For a 16-way cache, MID = 8 in LRU stack Medium – Insert at LRU-4(n/4) Position

Eg: For a 16-way cache, LRU-4 = 4 in LRU stack High – Insert at LRU Position

HPCA-13

Outline

Feedback Directed Prefetching Metrics and How to collect How to adapt

Results

Evaluation Methodology

Execution-driven Alpha simulator Aggressive out-of-order superscalar processor 1 MB, 16-way, 10-cycle unified L2 cache 500-cycle minimum main memory latency Detailed memory model

Prefetchers Modeled: Stream Prefetcher tracking 64 different streams Global History Buffer Prefetcher (in paper) PC-based Stride Prefetcher (in paper)

HPCA-13

Results: Adjusting Only Aggressiveness

4.7% higher avg IPC over the Very Aggressive configuration Most of the performance losses have been eliminated

HPCA-13

Results: Adjusting Only Cache Insertion Policy

5.1% better than inserting prefetches in MRU position 1.9% better than inserting prefetches in LRU-4 position

No PrefetchingLRULRU-4MIDMRUDynamic Insertion

Very Aggressive Prefetcher

HPCA-13

Results: Putting it all together (FDP)

6.5% IPC improvement over Very Aggressive configuration Performance losses converted to performance gains!

11%13%

HPCA-13

BPKI - Memory Bus Accesses per 1000 retired Instructions Includes effects of L2 demand misses as well as pollution

induced misses and prefetches

FDP significantly improves bandwidth efficiency

6.5% higher performance and18.7% less bandwidth

Bandwidth Impact

No. Pref. Very Cons Mid Very Aggr FDP

IPC 0.85 1.21 1.47 1.57 1.67

BPKI 8.56 9.34 10.60 13.38 10.88

13.6% higher performance with similar bandwidth usage

HPCA-13

Hardware Cost

Total hardware cost 20784 bits = 2.54 KB Percentage area overhead compared to baseline

1MB L2 cache 2.5KB/1024KB = 0.24% NOT on the critical path

pref-bits for L2 cache 16384 blocks 16384 bits

Pollution Filter 4096 entries * 1bit 4096 bits

16-bit counters 11 counters 176 bits

pref-bits for MSHR 128 entries 128 bits

HPCA-13

Outline

Feedback Directed Prefetching Metrics and collecting this information in

Hardware How to adapt

Results Conclusions

Contributions Comprehensive and low-cost feedback mechanism

for hardware prefetchers Uses

Prefetcher Accuracy Prefetcher Lateness Prefetcher-caused Cache Pollution

Adapts Aggressiveness Cache Insertion Policy for prefetches

6.5% higher performance and 18.7% less bandwidth compared to Very Aggressive Prefetching

Eliminates negative impact of prefetching Applicable to any data prefetch algorithm

HPCA-13

Questions?

HPCA-13

Backups

HPCA-13

FDP vs Prefetch Cache

Prefetch Caches eliminate prefetcher induced cache pollution

However, prefetches are now limited to the size of the prefetch cache

5.3% higher perf. than Very Aggr.+32KB Within 2% of Very Aggr.+64KB Memory bandwidth of FDP is 16% less than

32KB and 9% less than 64KB.

Performance on Other Prefetch algorithms Global History Buffer Prefetcher

20.8% less memory bandwidth than very aggressive with similar perf.

9.9% better performance than middle-of-the-road with similar bandwidth usage

PC-based Stride Prefetcher 4% better performance than the very aggressive 24% reduction in bandwidth usage

HPCA-13

IPC Performance

Dynamic Prefetcher Accuracy

Prefetch Lateness

Pollution Filter

Thresholds

Prefetches Sent

Distribution of dynamic aggressiveness level

Distribution of insertion position of prefetched blocks

Effect of FDP on memory bandwidth consumption

Performance of Prefetch cache vs FDP

Bandwidth consumption of prefetch cache vs. FDP

Effect of FDP on GHB

Effect of FDP on GHB(Bandwidth)

Effect of varying L2 size and memory latency

IPC on other benchmarks

BPKI on other benchmarks

Feedback Directed Prefetching Santhosh Srinath Onur Mutlu Hyesoon Kim Yale N. Patt §¥ ¥ §

Documents

Theory of Elasticity ,L.S. Srinath

Srinath sports quiz prelims

Context-Aware Prefetching at the Storage Server - USENIX · Context-Aware Prefetching at the Storage Server ... WWee implem perimental evaluation, we use th orage prefetching algorithm

Many-Thread Aware Prefetching Mechanisms for GPGPU ...hyesoon/lee_micro10.pdf · ern general-purpose GPU (GPGPU) systems employ just two: massive ﬁne-grained multithreading and

Spring 2010 Prof. Hyesoon Kim - Georgia Institute of ...hyesoon/spr10/lec_intro.pdf · • Prof. Hyesoon Kim • Attending HPCA conference. • Will be back in the ... (Nintendo DS

Prefetching Techniques

3 dn srinath-ge_complete-anaesthesiology_ncas_2011

Energy Efficient Prefetching and Caching

Informed Mobile Prefetching

Pis: Sudhakar Yalamanchili Hyesoon Kim Richard Vuduc...Pis: Sudhakar Yalamanchili Hyesoon Kim Richard Vuduc . 2 Web Search Networks Data Analytics Network Security Chip Design Bioinformatics

0 Srinath Horo

Fall 2011 Prof. Hyesoon Kim - gatech.edu

Ethernet oam-tutorial srinath-beldona

Informed Prefetching and Caching

Prefetching Lecture

Srinath Slides

Pis: Sudhakar Yalamanchili Hyesoon Kim Richard Vuducsynergy.cs.vt.edu/2015-nsf-xps-workshop/reports/Hyesoon_Kim_21-xps_kim.pdfPis: Sudhakar Yalamanchili Hyesoon Kim Richard Vuduc

Sunpyo Hong, Hyesoon Kim

Temporal Ancestry Prefetching - Research

Improving Index Performance through Prefetching