S.P.E.C CPU95

8/9/2019 S.P.E.C CPU95

1/6

SPEC 95

FAIRFAX, VA, August 21, 1995 -- The Standard Performance Evaluation Corp. (SPEC) announces the

release of the SPEC95 benchmark suites, the latest version of the worldwide standard for measuring

and comparing computer performance across different hardware platforms. SPEC95 was developed by

SPEC's Open Systems Group (OSG), which includes more than 30 leading computer vendors, systems

integrators, publishers and consultants throughout the world.

SPEC95 is a software benchmark product produced by the Standard Performance Evaluation Corp.

(SPEC), a nonprofit group of computer vendors, systems integrators, universities, research

organizations, publishers and consultants throughout the world. It was designed to provide measures of

performance for comparing compute intensive workloads on different computer systems. SPEC95

contains two suites of benchmarks: CINT95 for measuring and comparing compute- intensive integer

performance, and CFP95 for measuring and comparing computeintensive floating point performance.

"Computer systems technology evolves so rapidly that we must provide new benchmark suites every

two to three years to ensure a level playing field," says Kaivalya M. Dixit, SPEC president. "SPEC92

was a great success, but it is time to make the transition to standardized benchmarks that reflect the

advances in chip technologies, compilers and applications that have taken place over the last three

years; those benchmarks constitute SPEC95."

SPEC95 comprises two sets (or suites) of benchmarks: CINT95 for compute-intensive integer

performance and CFP95 for compute-intensive floating point performance. The two suites provide

component-level benchmarks that measure the performance of the computer's processor, memory

architecture and compiler. SPEC benchmarks are selected from existing application and benchmark

source code running across multiple platforms. Each benchmark is tested on different platforms to

obtain fair performance results across competing hardware and software systems.

SPEC95 is the third major version of the SPEC benchmark suites, which in 1989 became the first

widely accepted standard for comparing computeintensive performance across various architectures.

The new release replaces SPEC92, which will be gradually phased out between now and June 1996,

when SPEC will stop publishing SPEC92 results and stop selling the benchmark suite. Performance

results from SPEC95 cannot be compared to those from SPEC92, since new benchmarks have been

added and existing ones changed.

8/9/2019 S.P.E.C CPU95

2/6

"Compiler writers have learned to optimize performance for individual SPEC92 benchmarks," says Jeff

Reilly, SPEC95 release manager. "The best way to avoid these benchmark specific optimizations is to

develop new benchmark suites." SPEC95 builds on the lessons learned from the SPEC89 and

SPEC92 suites, according to Reilly. The new benchmarks were analyzed to ensure that they are as

resistant as possible to compiler optimizations that might not translate into realworld performance

gains. Improvements to the suites include longer run times and larger problems for benchmarks, more

application diversity, greater ease of use, and standard development platforms that will allow SPEC to

produce additional releases for other operating systems.

A Sun SPARCstation 10/40 with 128 MB of memory was selected as the SPEC95 reference machine

and Sun SC3.0.1 compilers were used to obtain reference timings on the new benchmarks. By

definition, the SPECint95 and SPECfp95 numbers for the Sun SPARCstation 10/40 are both "1."

The Metrics for Measurement

SPEC95 rules permit both baseline and optimized results for CINT95 and CFP95 suites. The baseline

rules restrict the number of optimizations that can be used for performance testing. In general, SPEC95

rules are more restrictive in regard to optimizations than the SPEC92 rules. Baseline metrics are

mandatory for all reported results; reporting of optimized results is optional.

SPEC95 also allows performance to be measured for both speed and throughput (rate). Speed metrics

such as SPECint95 measure how fast a computer completes a single task. Rate metrics such as

SPECint_rate95 measure how many tasks a computer can accomplish in a certain amount of time.

SPEC95 measures rate performance for single processors, symmetric multi processor systems and

cluster systems.

The CINT95 suite, written in C language, contains eight CPUintensive integer benchmarks. It is used to

measure and calculate the following metrics:

SPECint95

The geometric mean of eight normalized ratios (one for each integer benchmark) when

compiled with aggressive optimization for each benchmark.

SPECint_base95

The geometric mean of eight normalized ratios when compiled with conservative optimization

for each benchmark.

SPECint_rate95

8/9/2019 S.P.E.C CPU95

3/6

The geometric mean of eight normalized throughput ratios when compiled with aggressive

optimization for each benchmark.

SPECint_rate_base95

The geometric mean of eight normalized throughput ratios when compiled with conservative


The CFP95 suite, written in FORTRAN language, contains 10 CPU-intensive floating point

benchmarks. It is used to measure and calculate the following metrics:

SPECfp95

The geometric mean of 10 normalized ratios (one for each floating point benchmark) when

compiled with aggressive optimization for each benchmark.

SPECfp_base95

The geometric mean of 10 normalized ratios when compiled with conservative optimization for

each benchmark.

SPECfp_rate95

The geometric mean of 10 normalized throughput ratios when compiled with aggressive


SPECfp_rate_base95

The geometric mean of 10 normalized throughput ratios when compiled with conservative


Vendor Reporting

Initial results for systems from six vendor companies are included with this release. Additional results

will be reported in the next issue of the SPEC Newsletter, scheduled for publication at the end of

September. SPEC members are being encouraged to report SPEC95 results on older platforms to

provide an historical perspective for the new results.

Availability

SPEC95 (CINT95 and CFP95) is available on CD-ROM from SPEC's administrator, the National

Computer Graphics Association (NCGA). The cost is $600 for new customers, $300 for new university

customers, $300 for current SPEC92 licensees and $150 for current university licensees.

8/9/2019 S.P.E.C CPU95

4/6

SPEC is a non-profit corporation formed to establish, maintain and endorse a standardized set of

relevant benchmarks that can be applied to the newest generation of high-performance computers.

Included in its membership are the Open Systems Group (OSG); OSG Associates, consisting of

leading universities and research facilities; the HighPerformance Group (HPG); and HPG Associates.

Benchmarks

Q3: What does the "C" in CINT95 and CFP95 stand for?A3: In its product line, SPEC uses "C" to denote a "component-level" benchmark and "S" to denote a"system level" benchmark. CINT95 and CFP95 are component-level benchmarks.

Q4: What components do CINT95 and CFP95 measure?A4: Being compute-intensive benchmarks, they emphasize the performance of the computer'sprocessor, the memory architecture and the compiler. It is important to remember the contribution of

the latter two components; performance is more than just the processor.

Q5: What component performance is not measured by CINT95 and CFP95?A5: The CINT95 and CFP95 benchmarks do not stress other computer components such as I/O (diskdrives), networking or graphics. It might be possible to configure a system in such a way that one ormore of these components impact the performance of CINT95 and CFP95, but that is not the intent ofthe suites.

Q6: What is included in the SPEC95 package?A6: SPEC provides the following in its SPEC95 package: SPEC95 tools for compiling, running andvalidating the benchmarks, compiled for a variety of operating systems; source code for the SPEC95tools, to allow the tools to be built for systems not covered by the precompiled tools; source code forthe benchmarks; tools for generating performance reports; run and reporting rules defining how thebenchmarks should be used to produce standard results; and SPEC95 documentation.

The initial offering of SPEC95 will have tools for most UNIX operating systems. Additional products forother operating systems (Windows NT, OpenVMS, etc.) will be released as later products if SPECdetects enough demand. All of this will be shipped on a single CD-ROM disk.

Q7: What does the SPEC95 user have to provide?A7: The user must have a computer system running a UNIX environment with a compiler installed anda CD-ROM drive. Approximately 150 MB will be needed on a hard drive to install SPEC95. It is alsoassumed that the system has at least 64 MB of RAM to ensure that the benchmarks remain compute-intensive (SPEC is assuming this will be the standard amount of desktop memory during the life of thissuite).

Q8: What are the basic steps in running the benchmarks?A8: Installation and use are covered in detail in the SPEC95 User Documentation. The basic steps areas follows: Install SPEC95 from media. Run the installation scripts specifying your operating system.

Compile the tools, if executables are not provided in SPEC95. Determine what metric you wish to run.Create a configuration file for that metric. In this file, you specify compiler flags and other systemdependent information. Run the SPEC tools to build (compile), run and validate the benchmarks. If theabove steps are successful, generate a report based on the run times and metric equations.

Q9: What source code is provided? What exactly makes up these suites?A9: CINT95 and CFP95 are based on compute- intensive applications provided as source code.CINT95 contains eight applications written in C that are used as benchmarks:

8/9/2019 S.P.E.C CPU95

5/6

099.go -- Artificial intelligence; plays the game of "Go"124.m88ksim -- Moto 88K chip simulator; runs test program126.gcc -- New version of GCC; builds SPARC code129.compress -- Compresses and decompresses file in memory130.li -- LISP interpreter132.ijpeg -- Graphic compression and decompression134.perl -- Manipulates strings (anagrams) and prime numbers in Perl

147.vortex -- A database program

CFP95 contains 10 applications written in FORTRAN that are used as benchmarks:

101.tomcatv -- A mesh-generation program102.swim -- Shallow water model with 1024 x 1024 grid103.su2cor -- Quantum physics; Monte Carlo simulation104.hydro2d -- Astrophysics; Hydrodynamical Navier Stokes equations107.mgrid -- Multi-grid solver in 3D potential field110.applu -- Parabolic/elliptic partial differential equations125.turb3d -- Simulates isotropic,homogeneous turbulence in a cube141.apsi -- Solves problems regarding temperature, wind, velocity and distribution of pollutants145.fpppp -- Quantum chemistry146.wave5 -- Plasma physics; electromagnetic particle simulation

Benchmarks exist to act as a reference point for comparison. It's the same reason that EPA gas mileageexists, although probably no driver in America gets exactly the EPA gas mileage. If you understand whatbenchmarks measure, they're useful. It's important to know that CINT95 and CFP95 are CPU-focused andnot system-focused benchmarks. These CPU benchmarks focus on only one portion of those factors thatcontribute to applications performance. A graphics or network performance bottleneck within an application,for example, will not be reflected in these benchmarks. Understanding your own needs helps determine therelevance of the benchmarks.

Q14: Which SPEC95 metric should be used to determine performance?A14: It depends on your needs. SPEC provides the benchmarks and results as tools for you to use. Youneed to determine how you use a computer or what your performance requirements are and then choosethe appropriate SPEC benchmark or metrics.

A single user running a compute-intensive integer program, for example, might only be interested inSPECint95 or SPECint_base95. On the other hand, a person who maintains a machine used by multiplescientists running floating point simulations might be more concerned with SPECfp_rate95 orSPECfp_rate_base95.

8/9/2019 S.P.E.C CPU95

6/6

Types & Size of Operands

How is the type of an operand designated? Normally, encoding in the opcodedesignates the type of an operandthis is the method used most often. Alternatively,the data can be annotated with tags that are interpreted by the hardware.

These tags specify the type of the operand, and the operation is chosen accordingly.Computers with tagged data, however, can only be found in computermuseums.Let's start with desktop and server architectures. Usually the type of an operandinteger, single-precision floating point, character, and so oneffectivelygives its size. Common operand types include character (8 bits), half word (16bits), word (32 bits), single-precision floating point (also 1 word), and doubleprecisionfloating point (2 words). Integers are almost universally represented asB-14 Appendix B Instruction Set Principles and Examplestwo's complement binary numbers. Characters are usually in ASCII, but the 16-bit Unicode (used in Java) is gaining popularity with the internationalization ofcomputers. Until the early 1980s, most computer manufacturers chose their ownfloating-point representation. Almost all computers since that time follow thesame standard for floating point, the IEEE standard 754. The IEEE floating-pointstandard is discussed in detail in Appendix I.

Some architectures provide operations on character strings, although suchoperations are usually quite limited and treat each byte in the string as a singlecharacter. Typical operations supported on character strings are comparisonsand moves.For business applications, some architectures support a decimal format,usually calledpacked decimalorbinary-coded decimal4 bits are used toencode the values 0-9, and 2 decimal digits are packed into each byte. Numericcharacter strings are sometimes called unpacked decimal, and operationscalledpackingand unpackingare usually provided for converting back andforth between them.One reason to use decimal operands is to get results that exactly match decimalnumbers, as some decimal fractions do not have an exact representation inbinary. For example, 0.1010 is a simple fraction in decimal, but in binary itrequires an infinite set of repeating digits: 0.0001100110011. . . 2. Thus, calculationsthat are exact in decimal can be close but inexact in binary, which can be a

problem for financial transactions. (See Appendix I to learn more about precisearithmetic.)Our SPEC benchmarks use byte or character, half-word (short integer), word(integer), double-word (long integer), and floating-point data types. Figure B.llshows the dynamic distribution of the sizes of objects referenced from memoryfor these programs. The frequency of access to different data types helps indeciding what types are most important to support efficiently. Should the computerhave a 64-bit access path, or would taking two cycles to access a doubleword be satisfactory? As we saw earlier, byte accesses require an alignment network:How important is it to support bytes as primitives? Figure B.ll uses memoryreferences to examine the types of data being accessed.In some architectures, objects in registers may be accessed as bytes or halfwords. However, such access is very infrequenton the VAX, it accounts for nomore than 12% of register references, or roughly 6% of all operand accesses in

these programs.

Documents

S.P.E.C CPU95