An Evaluation of the Oak Ridge National Laboratory Cray XT3

Abstract

In 2005, Oak Ridge National Laboratory (ORNL) received delivery of a 5294 processor Cray XT3. The XT3 is Cray's third-generation massively parallel processing system. The ORNL system uses a single-processor node built around the AMD Opteron and uses a custom chip—called SeaStar—for interprocessor communication. The system uses a lightweight operating system called Catamount on its compute nodes. This paper provides a performance evaluation of the Cray XT3, including measurements for micro-benchmark, kernel, and application benchmarks. In particular, we provide performance results for strategic Department of Energy applications areas including climate, biology, astrophysics, combustion, and fusion. Our results, on up to 4096 processors, demonstrate that the Cray XT3 provides competitive processor performance, high interconnect bandwidth, and high parallel efficiency on a diverse application workload, typical in the DOE Office of Science.

Analyzing the execution of sparse matrix-vector product on the Finisterrae SMP-NUMA system

“…The average improvement is 1.1%, 9.0%, 15.1%, and 17.2% for two, four, eight, and sixteen threads, respectively. Therefore, taking advantage of Itanium2 caches on FINISTERRAE is critical, because accesses to memory incur in a high penalty in comparison with other systems [11]. Table 3 shows the SpMV performance without optimizations and when the optimizations detailed previously (data and thread allocation, and locality improvement) are applied together.…”

Application of self organizing maps for investigating network latency on a broadcast-based distributed shared memory multiprocessor

“…In current high performance architectures, the ratio of F/ C is in the range of 0.5-1. For example, in Cray XT 3 (Alam et al, 2008), F = 4.8 Â 10 9 , and the links have a peak bandwidth of 7.6 GB/s. With small cache blocks and miss rate in the neighborhood of 10% or less (due to the fact that programmers are going to target and distribute their applications for maximum locality, thus most accesses on well behaved applications are going to fall in cache), the resulting T/R is in the range of 0.05-1.…”

On the Path to Enable Multi-scale Biomolecular Simulations on PetaFLOPS Supercomputer with Multi-core Processors

“…Cray provides a Message Passing Interface (MPI) communication based on MPICH version 1.2 that uses Portals for data transfer. Details of the system can be found in [12].…”