Comparison of Processor Performance of SPECint2006 Benchmarks of some Intel Xeon Processors

Abdul Kareem PARCHUR^* and Ram Asaray SINGH

Department of Physics and Electronics, Dr. H. S. Gour University, Sagar, India-470003

E-mail: kareemskpa@hotmail.com

^* Corresponding author: Phone: +91-9907048098

Abstract

High performance is a critical requirement to all microprocessors manufacturers. The present paper describes the comparison of performance in two main Intel Xeon series processors (Type A: Intel Xeon X5260, X5460, E5450 & L5320 and Type B: Intel Xeon X5140, 5130, 5120 & E5310).  The microarchitecture of these processors is implemented using the basis of a new family of processors from Intel starting with the Pentium 4 processor. These processors can provide a performance boost for many key application areas in modern generation. The scaling of performance in two major series of Intel Xeon processors (Type A: Intel Xeon X5260, X5460, E5450 & L5320 and Type B: Intel Xeon X5140, 5130, 5120 & E5310) has been analyzed using the performance numbers of 12 CPU2006 integer benchmarks, performance numbers that exhibit significant differences in performance. The results and analysis can be used by performance engineers, scientists and developers to better understand the performance scaling in modern generation processors.

Keywords

Processor Performance; Benchmarks; Moore’s Law.

Introduction

The Intel Xeon processor are Intel’s most advanced IA-32 architecture microprocessor, incorporating a host of new microarchitectural features including a 400MHz system bus, hyper pipelined technology, advanced dynamic execution, rapid execution engine, advanced transfer cache, execution trace cache, and Streaming Single Instruction Multiple Data (SIMD) Extensions 2 (SSE2). The performance of processors is rapidly changing due to its architecture amendment and deep submicron pipelines. Our earlier studies reveal the performance scaling, hotspots detection and variation of task completion time in Intel and AMD processors [1-5]. However, high performance is becomes critical requirement for all processor manufacturers.

The analysis presented in this paper examines the scaling of performance in different series of Intel Xeon processors (Type A: Intel Xeon X5260, X5460, E5450 & L5320 and Type B: Intel Xeon X5140, 5130, 5120 & E5310) which are fabricated for the requirement of the modern generation utility.

Material and Method

Processor Performance Trends

The performance of modern processors is rapidly increasing as both clock frequency and the number of transistors required for a given implementation grow. Moore’s Law says that the device density of the processor double in every 18 months.  Figure 1 shows the transistor count per die of processors introduced by Intel over the past 35 years [6-8]. The processor that performs a given task in the least amount of time has the highest performance. Increased performance implies reduced execution time; the performance of the processor is given by [9]:

Performance = IPC × Frequency (IPC-instructions per clock)

(1)

Figure 1. Scaling transistors. The number of transistors is expected to continue to double about every two years, in accordance with Moore's Law. Over time, the number of additional transistors will allow designers to increase the number of cores per chip. [Source from 10]

The performance of the processor depends on the following factors. Firstly, the processor performance equation indicates that a processor’s performance is measured in terms of how long it takes to execute a particular program (time/program). Second, this measure of time/program or execution time can be formulated as a product of three terms: instructions/program, cycles/instruction, and time/cycle. The first term indicates the total number of dynamic instructions that need to be executed for a particular program; this term is also referred to as instruction count. The second term indicates on average how many machine cycles are consumed to execute each instruction; typically this term is denoted as the CPI (cycles per instruction). The third term indicates the length of time of each machine cycle, namely, the cycle time of the machine.

(2)

Equation 2 is known as Iron Law of processor performance.

Benchmark

Benchmarks are used for the performance evolution of the processors. There are different types of benchmarks available. Among all SPEC, HINT, and TPC are most important and popular benchmarks for performance evolution. SPEC is a non-profit corporation formed to establish, maintain, and endorse a standardized set of benchmarks. SPEC’s member-ship includes computer hardware and software vendors, leading universities, and research facilities worldwide. SPEC CPU2006 is designed to provide a comparative measure of compute-intensive performance across a range of hardware. Comprised of two suites of benchmarks, SPEC CPU2006 gauges compute-intensive integer performance with CINT2006 and measures floating-point performance with CFP2006. CINT2006 and CFP2006 results are presented as ratios, which are calculated using a reference time determined by SPEC and the runtime of the benchmark higher scores indicate better performance [11].

The SPEC CPU2006 suite contains 18 floating-point programs (Some programs are written in C and some in FORTRAN) and 13 integer programs (8 written in C, 4 in C++ and 1 in ANSI C). Table 1 and Table 2 provide a list of the benchmarks in SPEC CPU2006 suite. The SPEC CPU2006 benchmarks replace the SPEC89, SPEC92, SPEC95 and SPEC CPU 2000 benchmarks [12-15].

Table 1. The CINT 2006 Suite Benchmarks

S. No	Integer Benchmark	Language	Description
1	400.perlbench	C++	PERL Programming Language
2	401.bzip2	C	Data Compression
3	403.gcc	C	C Language Optimizing Compiler
4	429.mcf	C	Combinatorial  Optimization
5	445.gobmk	C	Artificial Intelligence : Game  Playing
6	456.hmmer	C	Search a Gene Sequence  Database
7	458.sjeng	C	Artificial Intelligence : Chess
8	462.libquantum	C	Physics / Quantum Computing
9	464.h264ref	C	Video Compression
10	471.omnetpp	C++	Discrete Event Simulation
11	473.astar	C++	Path – Finding Algorithm
12	483.xalancbmk	C++	XSLT Processor

Table 2. The CFP2006 Suite Benchmarks

S. No	Floating Point Benchmark	Language	Description
1	410.bwaves	Fortran – 77	Computational Fluid Dynamics
2	416.gamess	Fortran	Quantum Chemical Computations
3	433.milc	C	Physics /  Quantum Chromo Dynamics
4	434.zeusmp	Fortran – 77	Physics / Magneto Hydro Dynamics
5	435.gromacs	C/Fortran	Chemistry / Molecular Dynamics
6	436.cactusADM	C / Fortran-90	Physics / General Relativity
7	437.leslie3d	Fortran – 90	Computational Fluid Dynamics
8	444.namd	C++	Scientific, Structural Biology, Classical Molecular Dynamics Simulation.
9	447.dealII	C++	Solution of Partial Differential Equations using the Adaptive Finite Element Method.
10	450.soplex	C++	Simplex Linear Programming Solver
11	453.povray	C++	Computer Visualization / Ray Tracing
12	454.calculix	C/Fortran-90	Structural Mechanics
13	459.GemsFDTD	Fortran-90	Computational Electromagnetic
14	465.tonto	Fortran-95	Quantum Crystallography
15	470.lbm	C	Computational Fluid Dynamics
16	481.wrf	C/Fortran – 90	Weather Processing
17	482.sphinx3	C	Speech Recognition

Results and Discussion

In this study we utilize the integer benchmarks from the newly released SPEC CPU2006 suite [16] for the performance comparison of Intel Xeon processors (Type A: Intel Xeon X5260, X5460, E5450 & L5320 and Type B: Intel Xeon X5140, 5130, 5120 & E5310) under the same operating conditions. By using their performance numbers and frequency, we have calculated Task Completion time, and plotted graphs in between Task Completion Time (s) and Core Clock (ns) for Type A: Intel Xeon X5260, X5460,  E5450 &  L5320 and Type B: Intel Xeon X5140, 5130, 5120 & E5310 Processors. The Scalability and comparison of Intel Xenon series Processors is shown in figure 2. Extrapolation of the runtime trend lines down to zero core clock period gives basis for useful interpretation of system behaviour [17].

We have used the “Core Clock Period”, or “Clock Cycle” as the value characterizing the particular processor speed grade, expressed in nanoseconds (ns). The conversion formula is very simple: 1000/f [MHz] = clock period [ns]. Processor Performance Equation is calibrated using the expression:

Y = [Processor time]     = [Processor clock cycles for a program] × [Clock cycle time] = Ax.

(3)

The proportionality coefficient A is simply the number of processor clocks it takes to perform the task. The total number of instructions in the particular task (Instruction Count, IC) is known, and then dividing this coefficient by IC will give us the well-known microarchitectural parameter called CPI, “Clocks per Instruction”:

A = [Processor clock cycles for a program] = IC × CPI

(4)

The performance of the processor is calculated by using the relation:

Performance = Core Utilization Time % / Memory Wait Time %

(5)

The increase in performance of processors in different series of Intel Xeon processors (Type A: Intel Xeon X5260, X5460, E5450 & L5320 and Type B: Intel Xeon X5140, 5130, 5120 & E5310) was studied by using performance numbers published from SPEC CPU2006 Benchmark suit and results are shown in Figure 2.

Figure 2. The benchmark runtime vs. core clock period shows scaling of performance of Type A: Intel Xeon X5260, X5460, E5450 & L5320 and Type B: Intel Xeon X5140, 5130, 5120 & E5310 series processors, Extrapolation of the runtime trendlines down to zero core clock period gives basis for useful interpretation of system behavior.

The scaling of memory wait time and core time in Type A: Intel Xeon X5260, X5460, E5450 & L5320 and Type B: Intel Xeon X5140, 5130, 5120 & E5310 Series Processors are calibrated. It shows that Type A: Intel Xeon X5260, X5460, E5450 & L5320 Series processors shows maximum performance by showing its least memory wait time, it shows 96.43% core utilize time and 3.573 % memory wait time whereas Type B: Intel Xeon X5140, 5130, 5120 & E5310 Series Processors shows 89.07% core utilize time and 10.93 % memory wait time on SPECint 2006 Benchmark suit shown in Figure 3.

Figure 3. The comparison of memory wait time and core utilization time in Type A: Intel Xeon X5260, X5460, E5450 & L5320 and Type B: Intel Xeon X5140, 5130, 5120 & E5310 series processors.

Performances of the Type A and Type B series processors are compared by normalizing the performance of Type B Series Processor, which predicts that the performance of Type A series processors is 3.31 times faster as compared to Type B series processors (Figure 4). This method is so reliable to compare the performance of the modern processors.

Figure 4. The scaling of normalized performance in Type A: Intel Xeon X5260, X5460, E5450 & L5320 and Type B: Intel Xeon X5140, 5130, 5120 & E5310 series processors.

Contrary to prior work we not only quantify the performance prediction of the processors, but also evaluate scalability of the Memory Wait Time which degrades the performance of the processor by using simple statistical correlation techniques [1-5]. This analysis is more useful to performance engineers, scientists and developers to better understand how the performance scaling can be exploited in modern generation processors.

Disclaimer

All the observations and analysis done in this paper on SPEC CPU2006int Benchmarks are the author’s opinions and should not be used as official or unofficial guidelines from SPEC in selecting benchmarks for any purpose.  This paper only provides guidelines for performance engineers, academic users, scientists and developers to better understand the performance scaling in modern generation processors and to choose a subset of benchmarks the need be.

Acknowledgements

The author (Abdul Kareem PARCHUR) gratefully acknowledges financial support of UGC for a meritorious research fellowship.

References

1. Parchur A.K., Singh R.A., Performance Scaling of Individual SPEC INT 2006 Results for AMD Processors, Leonardo Electronic Journal of Practices and Technologies, 2009, 8(14) p.65-72.

2. Parchur A.K., Singh R.A., TCT Analysis of 0.2 ns Core Clock Series Processors, GESJ: Computer Science and Telecommunications, 2010, 26(3), p. 31-39.

3. Parchur A.K., Noorbasha F., Singh R.A., Detection of Weak Spots in Benchmarks Memory Space by using PCA and CA, Leonardo Electronic Journal of Practices and Technologies, 2010,16, p.43-52.

4. Parchur A.K., Singh R.A., Performance Scaling of Individual SPECint2006 results for Intel Xeon, e-Journal of Science & Technology, 2009, 4(2), p. 55-60.

5. Parchur A.K., Noorbasha F., Singh R.A., Study the Task completion Time of the Benchmarks @1GHz, 2GHz and 3GHz Processors, e-Journal of Science & Technology, 2010, 5(2), p.15-22.

6. Tuimi I., The Lives and Death of Moore's Law, First Monday, 2002, Oct 11.

7.  www.dell.com/powersolutions [accessed on August-2009].

8. Lilja D.J., Measuring Computer Performance A Practitioner's Guide, Cambridge University Press, New York, NY, 2000.

9. Understanding Processor Performance, A White paper, August 24, 2001, Online Available at: http://www.amd.com [accessed on August-2009].

10. From a Few Cores to Many: A Tera-scale Computing Research Review, A White Paper, Available at: www.intel.com [accessed on August 2009].

11. Henning J.L., SPEC CPU2000: Measuring CPU performance in the new millennium, IEEE Computer, July 2000, p. 28-35.

12. Standard performance evaluation corporation (SPEC), Benchmarks, http://www.spec.org [accessed on August-2009].

13. SPEC CPU2000 Press Release FAQ, Available at: http://www.spec.org/osg/cpu2000/press/faq.html [accessed on August-2009]. 

14. Osowski A.K., Lilja D., MinneSPEC: A new SPEC benchmark workload for simulation- based computer architecture research, Computer Architecture Letters, June 2002, p.1-7.

15. Ye D., Ray J., Harle C., Kaeli D., Performance Characterization of SPEC CPU2006 Integer Benchmarks on x86-64 Architecture, IEEE Computer Journal, 2006, p.120-127. [DOI: 10.1109/IISWC.2006.302736].

16. Standard Performance Evaluation Corporation, SPEC CPU benchmark suite, Available at: http://www.specbench.org/osg/cpu2006, [accessed on August-2009].

17. Predtechenski A., AMD A Method for Benchmarks Analysis, Online, Available at: http://www.spec.org/events/specworkshop/abs.html, 2006 [accessed on August-2009].