A Survey about Performance Counters, Libraries and Tools

Joseph Bryant Manzano Franco

Agenda

Introduction W3H: The Why, The What, The When, and The How

Hardware Performance Libraries Performance Application Programming Interface

(PAPI) Performance Counters Libraries (PCL)

Visualization Tools TAU: An example of a data collector KOJAK: Semi automatic instrumentation tool VAMPIR: An example of a script language PE: The All levels approach

Introduction

Program Optimization

Algorithm Optimization

Other ubiquitous optimizations

Architecture Optimizations

Search for the most effective algorithms and data structures

Consider common architecture features such cache structures

Apply architecture specific characteristic (PIM instructions, atomic load and stores, massive memory allocations, etc)

Data Collection

Data Analysis

Identify and solve unexpected problems with the interaction between hardware and software (memory and network bottlenecks, false sharing, poor cache management, etc)

IntroductionThe Why

Data Collection Data Analysis

High Level Library Functions

Performance Counters

Simulation environments

Easy to use and available on almost all libraries. Restricted and intrusiveCompose of timing function and clever data manipulation

Complete control over the environment including hardware, memory hierarchies and application code.Development is long for new architecturesSteep learning curve

Easy to use (especially with high level wrappers)Provides a range of measurements and is less intrusive

Manual Analysis

Automatic Statistical Analysis

Visualization Tools

Simple, but limited in its useProne to human error

Organize the data in a suitable formatStill need to deal with numbers

Graphical representation of data or its properties. Easy to identify trends even in large sets of data

Introduction:The What

Performance Counters

Special Registers that are present in an specific architecture

Designed to count architectural events

• An event is defined as an action that the hardware takes• Predefined• Examples: cache misses / hits, TLB misses / hits, context switches, cache invalidations, total instructions, etc

Sun Ultra SPARC Two 32 bit registers called PIC (Performance Instrumentation Counters). User control restricted

Pentium Pro Two 40 bit registers called PerfCrt0/1. User control available

Introduction: The When

Date Machine/Author Method of reading/Document

1966 Don Widring Initial Metering Design

~1970 GE 645 Multics

1979 Honeywell 6180 Yellow Submarine

1983 Cray-XM User Accessible Registers

Late 80 / early 90 IBM 3090 Mainframes, First generation IBM RS/6000

Restricted and Confidential

1992 First Alpha Chip (DEC) Uprofile, kprofile or IPROBE

1993 Pentium Not documented and embedded in the MSR

4

Introduction:The How

Example: Ultra SPARC Architecture

Two counters - 32 bits each

Event that are being counted: Number of Instructions (pic0), and Cache invalidations (pic1)

CPU CPU

$ $

Bus

pic0pic1 pic0

pic1load 0,s1

load 1,s2

inc s2

load 0,s1

load 0,s1

load 1,s2

add s1, s2, s1

store 0,s1

3210 10 43210 10

Agenda

Introduction W3H: The Why, The What, The When, and The How

Hardware Performance Libraries Performance Application Programming Interface

(PAPI) Performance Counters Libraries (PCL)

Visualization Tools TAU: An example of a data collector KOJAK: Semi automatic instrumentation tool VAMPIR: An example of a script language PE: The All levels approach

Hardware Performance Libraries

Performance Counters: Good idea, but only accessible to hardware experts.

Solution: High Level Wrappers. Usually written in C and Fortran. Easy to make them thread safe and to

integrate them in existent code. Examples:

Performance Application Programming Interface (PAPI)

Performance Counters Library (PCL)

Performance Application Programming Interface A high Level wrapper functions that includes a vast set of architectures

and events Available for Power3, Power4, Ultra SPARC II and III, all flavors of

Pentium, Itanium, AMD Athlon, etc. Well documented, stable and reliable programming interface. Goals of the PAPI project:

To provide a solid foundation for cross platform performance analysis tools

To present a set of standard definitions for performance metrics on all platforms

To provide a standardize API among users, vendors, and academics To be easy to use, well documented, and freely available (Excerpt obtained from the PAPI user guide)

PAPI is an effort of the Innovative Computer Laboratory (ICL) that is part of the Department of Computer Science at the University of Tennessee

PAPI

Platform PAPI_read() – PAPI 3.0 Altix (Itanium 2 -Madison Chip) 1357 Cycles/Call IBM Power 4 4034 Cycles/Call Itanium 2 (libpfm 2.0) 1606 Cycles/Call Pentium 3 (perfctr 2.4.5) 324 Cycles/Call Pentium 4 (perfctr 2.4.5) 401 Cycles/Call SGI R12k 3681 Cycles/Call Ultrasparc II 2150 Cycles/Call

High Level API

Kernel ExtensionsOperating System

Hardware Performance Counters

Low Level APIPortable Layer

Machine Dependent Layer

Substrate

Blo

ck D

iag

ram

Ove

rhea

d

PAPI:Terminology Native Events:

Defined as countable by an specific CPU. Machine dependent Hexadecimal value and a mask provided by PAPI libraries

Present Events: Predefined events. Events (or group of events) that are considered useful and

relative ubiquitous across architectures. A PAPI identifier is provided

Event List: A array of events (usually the consist of PAPI identifiers)

PAPI:Terminology High Level API:

A group of functions A single of list of events Access to Native Events is prohibited. Flexibility and performance is lost due to its easiness to

use Low Level API:

Another group of functions Multiple event list definitions and native events

interface. Only one event list can be running at any point in time

PAPI:Steps

Initialization of the PAPI library

Start the counters

Operate on the counters

Stop the counters

De-allocate any resource that has been allocated

#include <papi.h>#include <stdio.h>

#define NUM_EVENTS 2

int main(int argc, char **argv){ int Events[NUM_EVENTS] = { PAPI_TOT_INS, PAPI_TOT_CYC }; long_long values[NUM_EVENTS], val2[NUM_EVENTS]; int a= 0; int retval; retval = PAPI_library_init(PAPI_VER_CURRENT); PAPI_start_counters(Events, 2); PAPI_read_counters(values, 2); a++; PAPI_read_counters(values, 2); PAPI_read_counters(val2, 2); printf("The value of a is: %i \n", a); printf("The Coarse Instructions are: %10lld\n", values[0]); printf("The Coarse Cycles are: %10lld\n", (values[1])); printf("The Overhead Instructions are: %10lld\n", val2[0]); printf("The Overhead Cycles are: %10lld\n", (val2[1])); printf("The Total Instructions are: %10lld\n", (-val2[0] + values[0])); printf("The Total Cycles are: %10lld\n", (-val2[1] + values[1])); PAPI_stop_counters(values, 2); return 0;}

PAPI:Output

The value of a is: 1The Coarse Instructions are: 179The Coarse Cycles are: 641The Overhead Instructions are: 175The Overhead Cycles are: 395The Total Instructions are: 4The Total Cycles are: 246

ld [%fp-52],%l0 add %l0,1,%l0 st %l0,[%fp-52] add %fp,-32,%o0

Assembly Output of a++

The first access to produce a (L2) cache miss

PAPI:Extra Features Multithread safe and support Multiplexing where available Overflow control with thresholds Statistical Profiling and related functions Error detection and control features

Performance Counters Libraries

Another Example of High Level performance counters Events are classified (as in PAPI) as Memory Hierarchy events

(caches, TLB, memory, etc), Instructions (Instruction types, Instructions completed, etc), Status of Functional Units and rates and ratios.

It supports the Pentium architectures up to Pentium 4, the AMD Athlon / Duron, the IBM Power series up to Power 3-II, Alpha’s 21164 and 21264, SGI’s R10000 and R12000 and the UltraSPARC family of processors

PCL is available for C, C++ and Java PCL is an effort of Forschungszentrum Juelich GmbH and the

University of Applied Sciences Bonn-Rhein-Sieg from Germany and currently it is in its second version

PCL High Level API:

Similar to PAPI High Level API but the functions are different.

Events lists can be created in this API Access to predefine events only Recommended

Low Level API: Let to access the performance counters directly Not recommended

Handle: A single Data (usually an integer) that is used to uniquely

identify a set of resources. Used to provide a thread specific link to the resources (the

list of events)

PLC:Steps

#include <pcl.h>int main(int argc, char **argv){ int counter_list[2], a = 0; int ncounter; unsigned int mode; PCL_CNT_TYPE i_result_list[2]; PCL_FP_CNT_TYPE fp_result_list[2]; PCL_DESCR_TYPE descr; PCLinit(&descr); ncounter = 2; counter_list[0] = PCL_CYCLES; counter_list[1] = PCL_INSTR; mode = PCL_MODE_USER; PCLstart(descr, counter_list, ncounter, mode); a++; PCLstop(descr, i_result_list, fp_result_list, ncounter); printf("%f instructions in %f cycles\n", (double)i_result_list[1], (double)i_result_list[0]); PCLexit(descr); return 0;

}

Initialization of the PCL library

Start the counters

Operate on the counters

Stop the counters

De-allocate any resource that has been allocated

PLC:Differences with PAPI Nested function call enabled Rates and Ratios are function calls in PAPI

libraries Low Level API deals with native code as

PAPI’s Low level does but its used is not recommended in PCL

Agenda

Introduction W3H: The Why, The What, The When, and The How

Hardware Performance Libraries Performance Application Programming Interface

(PAPI) Performance Counters Libraries (PCL)

Visualization Tools TAU: An example of a data collector KOJAK: Semi automatic instrumentation tool VAMPIR: An example of a script language PE: The All levels approach

Visualization Tools

After gathering the information for the tools, how to present it to the user in the most efficient matter?

The visualization tools provide a good way to present trends in data across extensive data sets

Examples of Visualization tools: Tuning and Analysis Utilities Kit for Objective Judgement and Knowledge-based

Detection of Performance Bottlenecks VAMPIR / VAMPIRTrace Performance Evaluator

Tuning and Analysis Utilities (TAU)

Program and Performance analysis tool framework for high-performance parallel and distributed computing.

A suite of tools for static and dynamic analysis of programs written in C, C++, FORTRAN 77/90, Python, High Performance FORTRAN, and Java.

Instrumentation by functions The concept of Inclusive and Exclusive

With Time Exclusive time Refers to the time spent in the function minus all the

time spent on functions that has instrumented and called by this function

Inclusive time Total time of the function With Performance Counter

The same as time with the properties of that performance counter Supported extensions in C and FORTRAN: MPI and OpenMP Hardware Counters supported: PAPI and PCL

TAU Infrastructure

KOJAK

Kit for Objective Judgement and Knowledge-based Detection of Performance Bottlenecks

A complete infrastructure dedicated to find performance bottlenecks and application properties

Consists of the following components OpenMP Pragma And Region Instrumentor (OPARI)

(Redirect the OpenMP function call and directives toward wrappers that contains instrumentation information (POMP)) and PMPI

TAU (function instrumentation) Event Processing, Investigating and Logging (EPILOG)

runtime library (event oriented trace creator utility)

KOJAK

Extensive Performance Tool (EXPERT) (trace files analyzer search for low performing sections on them and classify them according to severity) uses the Event Analysis and Recognition Library (EARL)

CUBE (KOJAK’s Trace visualization tool) Trace transformations to different formats (to

VAMPIR trace format)

KOJAK Infrastructure

KOJAK Snapshots

KOJAK Snapshots

VAMPIR

A configurable visualization trace tool Converts trace information into a variety of graphical views:

Process State Display Statistics Display Timeline Display Communications Statistics Configured by using

Pull-down menus Configuration file

The displays can be related to the source code Zoom in and Zoom out Advance feature Defined trace format: VAMPIR-Trace (runtime library enhanced

with trace creation calls)

VAMPIR Infrastructure

Source Code

Guide Compiler Object Files

Linker

Guide Libraries

VAMPIRTrace Libraries

Executable

Config File

Trace File VAMPIR V

VAMPIR Snapshot

Performance Evaluator

Java Based Tool All level analysis of a program behavior:

Application Software level analysis Data / Algorithm Analysis

Operation System level analysis Thread context switching Thread scheduling

Hardware Level Analysis Memory Hierarchy

Used PMAPI performance counters (IBM proprietary)

Performance Evaluator Infrastructure

K42 Infrastructure

AIX OS

Others

Parser / Modifier

PE2 Visualization Tool

32

1

32

1

1 Trace Format File2 Map File3 Meta File

PE Trace Format PE2 Trace Format

Performance Evaluator:A Run Get Hardware Information from the

infrastructures (the source has been instrumented and the OS is collecting information also)

Create: Trace file (s) Trace records of a program

with short hand versions of events Map file Have static information about

functions, threads and other structures Meta file (s) Properties of a trace, records

type definitions and Map type definitions

Performance Evaluator:A Run Feed the files to the tool Visualize the information with graphs Contemplate the whole application behavior

since beginning to the end Complete GUI with the Eclipse Workbench Designed to work with several Multi Threaded

packages in C and Java OpenMP not supported

Thanks so much for your time

Questions? Comments?