Instrumentation and Measurement

CSci 599 Class PresentationShreyans Mehta

Abstract• Why Instrumentation and Measurement ?• Instrumentation Techniques• Resources• Data Analysis• Case Study: Paradyn

– Guiding Principles– System Overview– W3 Search Model

Why Instrumentation and Measurement ?

• Gathering data to improve the next execution of the program.

• Guiding scheduling decisions• Adapting to computations while in

execution

Instrumentation Techniques• Program Instrumentation Techniques

– Manual : Programmer inserted directives– Automatic : No direct user involvement

• Binary Rewriting• Dynamic Instrumentation

• Processor Instrumentation Techniques– Information includes timers, memory system

performance, processor usage, etc.– Available mostly through special registers or memory

mapped location.• Example : Pentium Pro provides performance data through

MSRs. These registers include 64 bit cycle clock and counts of memory read /write, L1 cache misses, pipeline flushes, etc.

– Hardware assisted trace generation.

• Operating System Instrumentation Techniques– Information includes behavior of virtual memory, file

system, file cache etc.– Instrumentation in the form of APIs for applications to

access these variables.• Network Instrumentation Techniques

– Ways of measuring• Passive

– Example: RMON protocol defines SNMP MIB variables to report traffic statistics over hubs and switches.

• Active– Example: Ping, NWS in grid style computing.

Data Storage Representation

• Scalars– Counters– Times

• Traces• Vector series

Resources

• Software Abstractions– Program Components – Code in Executions– Synchronization Objects– Other Software Abstractions

• Hardware Abstractions• Network Abstractions

Data Analysis

• Quantitative Performance• Automating Performance Diagnosis• Perturbation Analysis

The Paradyn Parallel Performance Measurement Tools

Case Study

Guiding Principles and Characteristics

• Scalability• Automate the search for performance problems• Provide well-defined data abstractions• Support heterogeneous environments• Support high level parallel languages• Open interfaces for visualization and new data

sources• Streamlined use

System Overview

• Basic Abstractions– Metric-focus grid – Time Histograms

• Components of the System– Main Paradyn Process

• Performance Consultant• Visualization Manager• Data Manager• User Interface Manager

– Paradyn daemons– External Visualization Processes.

Histogram VisualizationTable Visualization

Tabular Summary

CPU 3.0 4.0

Messages 117 81

Metric Manager

Instrumentation Manager

Metric Manager

InstrumentationManager

Visualization

Manager

User Interface Manager

Performance Consultant

Data Manager

Application ApplicationProcesses Processes

Visi Thread Visi Thread

Paradyn Daemon(s)

Paradyn

Dynamic Instrumentation• Dynamic Instrumentation Interface

– Metric Manager– Instrumentation Manager

• Points, Primitives and Predicates

addCounter(fooFlg, 1)

addCounter(fooFlg, 1)

Foo(){ …. ….}

SendMsg( dest, ptr, cnt, size){ …. ….}

if (fooFlg) startTimer(msgTme, ProcTime)

if (fooFlg) stopTimer(msgTme)

• Instrumentation generation– Base Trampolines– Mini-Trampolines

• Data Collection• Internal Uses of Dynamic Instrumentation

– Resource Discovery– Collection of dynamic mapping information for

HLL.

The W3 Search Model and the Performance Consultant

• Why ? Where ? When ?– The “Why” Axis

• Why is the application performing poorly ?– Potential performance problems are represented as hypotheses

and tests.– Hypotheses represent activities universal to all parallel

computations.– Hypotheses can be refined into more refined hypotheses using a

search hierarchy.– Tests are Boolean functions that evaluate the validity of a

hypotheses.– Tests are expressed in terms of a threshold and metrics

calculated by the Instrumentation Manager.

A sample “why” axis with several hypotheses

TopLevelHypotheses

SyncBottleNeck

HighSyncBlockingTimeFrequentSyncOperations

HighSyncContentionHighSyncHoldingTime

– The “Where” Axis• Where is the performance problem ?

– Pinpoints the problem specific to program components.– Each hierarchy in “where” axis has multiple levels, with the

leaf nodes being the instances of resources used by the application.

SyncObject

Semaphores Message SpinLock Barier

– The “When” Axis • When does the problem occur ?

– Represents periods of time during which performance problems can occur.

• The Performance Consultant– This module discovers performance problems

by searching the space defined by W3 Search Model.

– Fully automated search but also allows user to make manual refinements.

Open Visualization Interface

• Paradyn provides a simple library and RPC interface to access performance data in real-time.

• Visualization modules (visi’s) in Paradyn are external processes that use this library and interface.

• Currently provides visi’s for time-histograms, bar charts and tables.

Examples of Use

Conclusion

Computational grids are focused on high performance distributed computing. To achieve high performance, such systems need to provide tools that enable the programmer to realize the potential performance inherent in such a system.

References• Jeffery K. Hollingsworth and Bart Miller, “Instrumentation and

Measurement”, Chapter 14 of Grid: The Blueprint for a new computing infrastructure.

• Bart Miller, “The Paradyn Parallel Performance Measurement Tools”, http://www.cs.wisc.edu/~paradyn/papers/index.html