Standard Performance Evaluation Corporation

The Standard Performance Evaluation Corporation (SPEC) is an American nonprofit organization that aims to “produce, establish, maintain and endorse the standardized set of performance benchmarks for computers. [1]

SPEC was founded in 1988. [2] [3] SPEC benchmarks are widely used to evaluate the performance of computer systems; the test results are published on the SPEC website.

SPEC evolved into an umbrella organization encompassing four diverse groups; Graphics and Workstation Performance Group (GWPG), the High Performance Group (HPG), the Open Systems Group (OSG) and the newest Research Group (RG). More details are on their website; [1] .


Membership in SPEC is open to any interested company that is willing to commit to SPEC’s standards. It allows:

  • Participation in benchmark development
  • Participation in review of results
  • Complimentary software based on group participation

The list of members is available on SPEC’s membership page; [2] .

Membership Levels

  • Sustaining Membership requires payment and usually includes hardware or software companies.
  • SPEC “Associates” pays a reduced fee and typically include Universities.
  • SPEC “Supporting Contributors” are invited to participate in a single benchmark, and do not pay fees.

SPEC Benchmark Suites

The benchmarks aim to test “real-life” situations. There are several benchmarks testing Java scenarios, from simple computation (SPECjbb) to a full system with Java EE, database, disk, and network (SPECjEnterprise).

The SPEC CPU suite test CPU performance by measuring the run time of Several programs Such As the compiler GCC , the chemistry program gamess , and the weather program WRF . The various tasks are equally weighted; no attempt is made to weight them on their perceived importance. An overall score is based on a geometric mean.


SPEC benchmarks are written in a portable programming language (usually C , C #, Java or Fortran ), and the interested parties may compile the code using whatever compiler they prefer for their platform, but may not change the code. Manufacturers have been known to optimize their compilers to improve the performance of the various SPEC benchmarks. SPEC has rules that attempt to limit such optimizations.


In order to use a benchmark, a license has been purchased from SPEC; the costs vary from one test to one of several thousand dollars. This pay-for-license model might seem like a violation of the GPL as the benchmarks include software such as GCC that is licensed by the GPL. However, the GPL does not require software to be distributed for free, only those recipients are allowed to receive GPLed software that they receive; The license agreement is specifically designed to exempt items which are under “licenses that require free distribution”, and the files themselves are placed in a separate part of the overall software package.


Current Benchmarks

  • SPECapc for 3ds Max ™ 2011, performance evaluation software for systems running Autodesk 3ds Max 2011.
  • SPECapcSM for Lightwave 3D 9.6, performance evaluation software for running systems NewTek LightWave 3D v9.6 software.
  • SPEC CPU2006, combined performance of CPU, memory and compiler. COMP CPU2006 contains two benchmark suites: CINT2006 for measuring and comparing compute-intensive integer performance, and CFP2006 for measuring and comparing compute-intensive floating point performance. .
    • CINT2006 (” SPECint “), testing integer arithmetic, with programs such as compilers, interpreters, word processors, chess programs etc.
    • CFP2006 (” SPECfp “), floating point performance testing , with physical simulations, 3D graphics, image processing, computational chemistry etc.
  • SPEC CPUv6, The CPU Search Program seeks to encourage those outside the scope of the application of CPU-intensive benchmark suite, currently designated as SPEC CPUv6.
  • SPECjbb2013, evaluates the performance of Java by emulating a three-tier client / server system (with emphasis on the middle tier). The SPECjbb2013 benchmark has been developed from the ground up to measure performance based on the latest Java application features. Who are interested in Java server performance, including JVM vendors, hardware developers, Java application developers, researchers and members of the academic community.
  • SPECjEnterprise2010, a multi-tier benchmark for measuring the performance of Java 2 Enterprise Edition (J2EE) technology-based application servers. SPECjEnterprise2010 measures full-system performance for Java Enterprise Edition (Java EE) 5 or later application servers, databases and supporting infrastructure and expands the scope of the SPECjAppServer2004 benchmark.
  • SPECjms2007, Java Message Service Performance. SPECjms2007 is the first industry-standard benchmark for the performance of enterprise message-oriented middleware servers based on JMS (Java Message Service). It provides a standard workload and performance metrics for competitive product comparisons, as well as a framework for indepth performance analysis of enterprise messaging platforms.
  • SPECjvm2008, measuring basic Java performance of a Java Runtime Environment on a wide variety of both client and server systems. SPECjvm2008 is a benchmark for Java Runtime Environment (JRE), which contains several real life applications and benchmarks focusing on core Java functionality. The SPECjvm2008 workload mimics a variety of common general purpose application computations.
  • SPECpower_ssj2008 , evaluates the energy efficiency of server systems. SPECpower_ssj2008 is the first industry-standard SPEC benchmark that evaluates the power and performance characteristics of servers.
  • SPECsfs2008, NFS and SMB protocol access. SPECs benchmark designed to evaluate the speed and demand-handling capabilities of the servers utilizing the NFSv3 and SMB protocols.
  • SPECvirt_sc2013 (” SPECvirt “), evaluates the performance of datacenter servers used in virtualized server consolidation environments.
  • SPECapc, performance of several 3D-intensive applications
  • SPEC MPI2007, for evaluating performance of parallel systems using MPI ( Message Passing Interface ) applications.
  • SPEC OMP2001 V3.2, for evaluating performance of parallel systems using OpenMP ( ) applications.

Graphics and Workstation Performance

  • SPECviewperf® 12
  • SPECviewperf® 11, performance of an OpenGL 3D graphics system, tested with various rendering tasks from real applications
  • SPECwpc
  • SPECapcSM for 3ds Max ™ 2015
  • SPECapcSM for Maya® 2012
  • SPECapcSM for PTC Creo 2.0
  • SPECapcSM for Siemens NX 8.5
  • SPECapcSM for SolidWorks 2013

High Performance Computing, OpenMP, MPI, OpenACC, OpenCL

  • SPEC ACCEL, SPEC ACCEL tests performance with a suite of computationally intensive parallel applications running under the OpenCL and OpenACC APIs. The suite exercises the performance of the accelerator, host CPU, memory transfer between host and accelerator, support libraries and drivers, and compilers.
  • SPEC MPI2007, MPI2007 is SPEC’s benchmark suite for MPI-parallel evaluation, floating point, compute intensive performance across a wide range of cluster and SMP hardware. The suite consists of the MPIM2007 MPIL2007 and MPIM2007.
  • SPEC OMP2012, The successor to the OMP2001, designed for measuring performance using applications based on the OpenMP 3.1 standard for shared-memory parallel processing. OMP2012 also includes an optional metric for measuring energy consumption.

SPEC Tools

  • Server Efficiency Rating Tool (SERT). The US Environmental Protection Agency (EPA) is the first generation of the US Environmental Protection Agency (EPA) ENERGY STAR for Computer Servers program. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS OF A COMPREHENSIVE GUIDE TO SERIOUS INTERFACES AND THE USE OF A COMPREHENSIVE DIGITAL SYSTEM.
  • Driver Worklet Development Kit (WDK). Chauffeur was designed to simplify the development of workloads for measuring both performance and energy efficiency. Because Driver contains functions that are common to most workloads, they can focus on the current business logic of the application, and take advantage of Driver’s capabilities for configuration, run-time, data collection, validation, and reporting. Chauffeur was initially designed to meet the requirements of the SERT. However, SPEC recognized that the framework would also be useful for research and development purposes. The Driver Framework is now available as the Driver Worklet Development Kit (WDK). This kit can be used to develop new workloads (or “worklets” in Chauffeur terminology). Researchers can also use the WDK to configure workflows in different ways, in order to mimic the behavior of different types of applications. These features can be used in the development and assessment of new technologies such as power management capabilities.
  • PTDaemon. The SPEC PTDaemon software is used to control power analyzers in benchmarks which contain a power measurement component.

Retired Benchmarks

  • SPEC CPU2000
  • SPEC CPU95
  • SPEC CPU92
  • SPEC 2001
  • SPEC HPC2002
  • SPEC HPC96
  • SPECjAppServer2004
  • SPECjAppServer2002
  • SPECjAppServer2001
  • SPECjbb2005
  • SPECjbb2000
  • SPECjvm98
  • SPECmail2009
  • SPECmail2008
  • SPECmail2001
  • SPEC SDM91
  • SPEC SFS97_R1 3.0)
  • SPEC SFS97 (2.0)
  • SPECsip_infrastructure2011
  • SPEC virt_sc2010
  • SPECweb2009
  • SPECweb2005
  • SPECweb96
  • specweb99
  • SPECweb99_SSL


Representations of the environment, where the arguments are based on the concept of technical credibility, representativeness, or the “level playing field”. SPEC representatives are typically engineers with expertise in the areas being benchmarked. Benchmarks include “run rules”, which describe the conditions of measurement and documentation requirements. Results that are published on SPEC.


  1. Jump up^ “SPEC Frequently Asked Questions” . Retrieved 15 March 2010 .
  2. Jump up^ “The SPEC Organization” . Retrieved 15 March 2010 .
  3. Jump up^ “SPEC Membership” . Retrieved 15 March 2010 .

Leave a Reply

Your email address will not be published. Required fields are marked *

Copyright 2019
Shale theme by Siteturner