A framework for computer performance evaluation using benchmark sets

Krishnaswamy, Umesh; Scherson, Isaac D.

doi:10.1109/12.895853

Cited by 24 publications

(12 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Python Opcode counting, categorization, and timing were used for both. The first method is similar to the one outlined by [21], [7], [24] . The second method used was bytecode counting and timing similar to the method used by the JRAF-2 framework and associated publications [23], [22].…”

Section: Performance Vectormentioning

confidence: 99%

See 1 more Smart Citation

Resource selection and allocation for dynamic adaptive computing in heterogeneous clusters

Duselis

Cauich

Wang

et al. 2009

2009 IEEE International Conference on Cluster Computing and Workshops

View full text Add to dashboard Cite

This paper provides a framework for dynamic adaptive computing in heterogeneous clusters for computationally intensive applications. The framework considers a set of discoverable interconnected computational resources and either a parallel or sequential workload needing to be executed. An adaptive inclusion/exclusion algorithm is used to select the resources by using novel performance measurements and profiling techniques. Furthermore, contrary to a greedy approach where all the resources are seized for the workload application, our framework only harnesses the best fit resources measured against system-wide performance characterization, and is contingent upon the current workload definition. The intelligent selection of a subset of resources has proven to achieve better performance; especially in environments with a high level of heterogeneity where the characteristics of some resources may not achieve the best performance the cluster can provide. Additionally, this paper provides a novel analysis of the workload and cluster characteristics, exhibiting analytical starting points to be used in the resource selection.

show abstract

Section: Performance Vectormentioning

confidence: 99%

“…Let P be a matrix comprised of {P 1 , ..., P n } performance vectors, where each P k for k = {1,...,n} is the individual performance vector defined in [21].…”

Section: A System-wide Formalizationmentioning

confidence: 99%

Resource selection and allocation for dynamic adaptive computing in heterogeneous clusters

Duselis

Cauich

Wang

et al. 2009

2009 IEEE International Conference on Cluster Computing and Workshops

View full text Add to dashboard Cite

show abstract

“…Software performance estimation is a well-known instance of performance modelling [25]. The goal of software estimation techniques is to determine the time required for the execution of a certain task on a processor at an abstraction level that makes the estimation time reasonable without losing too much accuracy.…”

Section: Software Performance Estimationmentioning

confidence: 99%

The use of intelligent data analysis techniques for system-level design: a software estimation example

Bontempi

Kruijtzer

2003

Soft Computing

View full text Add to dashboard Cite

System-level designers typically rely on whitebox detailed descriptions of embedded systems in order to perform their design choices and optimisations. The white-box approach assumes that the complexity of the system can be managed by decomposing the global system into a set of simple and well-described components. This assumption is generally not true when non-linear interactions happen between the different subsystems. This paper explores, as an alternative, a black-box modelling methodology based on intelligent data analysis in order to support system-level designers in characterising the performance of embedded applications. The rationale behind this approach is that nowadays the designer can easily store large amount of data related to the system, and consequently discover relevant information by analysing them in a black-box fashion. As a case study, we address the performance modelling of software applications running on an embedded microprocessor. We introduce a data analysis method which, on the basis of a high-level characterisation of the software functionality and the hardware architecture, is able to predict the number of execution cycles on a embedded processor. We propose the adoption of a local learning technique (lazy learning), which proved already to be effective in previous works, to model the unknown input/output relation between the hardware/software parameters of the application and the number of execution cycles. Experiments with standard computational code (sorting, mathematical computation) and with a specific streaming algorithm for MPEG variable length decoding are presented to support this claim. IntroductionAs design complexity has grown and time-to-market requirements have shrunk drastically, both electronic industry and academia have begun to focus on possible solutions to the widening gap between the technology capacity (as predicted in the Moore's law) and the design productivity. All the latest efforts in design automation address the required paradigm shift in electronic system design [18,22]. Consider, for example, topics like the reuse of components [21], the raise of the level of abstraction in design and the wide adoption of virtual simulation methods [3].While these approaches, on one side, appear as powerful ways to deal with the complexity of the electronic design problem, on the other side present some intrinsic difficulties. These are related to the fact that every abstraction effort requires a designer to define the level of representation which is compatible with the goal and the constraints of his analysis. In other terms, as schematically represented in Fig. 1, reasoning at a higher level of abstraction guarantees a simpler representation, faster analysis and a larger impact on the final result but inevitably implies a deterioration of the accuracy.Finding the convenient trade-off is definitely an important task that a system-level methodology [18] is called upon to solve. Although this awareness is widely spread in the electronic design community, many techniques ...

show abstract

“…where i is the ideal execution time given by the manufacturer's specified latency for the instruction, and t i is the context-free, actual execution time found using the performance vector [1] , [2] .…”

Section: Code Structuresmentioning

confidence: 99%

Selection of Optimal Computing Platforms through the Suitability Measure

McMahon

Scherson

2007

Sixth International Symposium on Parallel and Distributed Computing (ISPDC'07)

View full text Add to dashboard Cite

Origin of the suitability measureOne of the most difficult tasks for designers of interstellar and interplanetary probes is selection of the most suitable computing platform. Several factors must be considered, including physical size, weight, power constraints, radiation hardness, cooling issues and computing capability. Providing high performance while satisfying the other constraints often results in tradeoffs such as reductions in performance to meet cooling and power supply restrictions. Because no quantifiable standards currently exist, the choices often rely upon educated guesses. The suitability measure will provide a method to quantify the match between a program and a machine.Contemporary performance analysis techniques are concerned primarily with runtime. Instead, the suitability measure quantifies how efficiently a computer executes a program. A machine that runs a program more slowly, but uses its resources more effectively, has a higher suitability than a faster machine that provides poor resource utilization. This makes the measure particularly well suited to spaceborne computing, embedded systems, and other cases where a balance must be struck among a collection of conflicting constraints.The suitability measure combines opcode-level performance metrics with instruction mix and control flow characteristics collected from static scans of compiled programs. It reveals a machine's efficiency at executing the mean instruction mix seen, over the set of all possible execution paths a program may follow. By approaching the problem from an efficiency perspective, time is factored out of the equations, yielding a time-independent measure. Instruction level suitabilitySuitability is a composite quantity combining instructional execution efficiency, resource utilization efficiency, and a relative load metric. Each of the terms is unitless. Instructional execution efficiency quantifies a computer's efficiency at executing a single type of instruction. The definition can be relaxed to allow the inclusion of instruction classes or high-level operations such as macros. Its accuracy is inversely proportional to the granularity of the instruction type. It is defined as . (1) a.

show abstract

A framework for computer performance evaluation using benchmark sets

Cited by 24 publications

References 27 publications

Resource selection and allocation for dynamic adaptive computing in heterogeneous clusters

Resource selection and allocation for dynamic adaptive computing in heterogeneous clusters

The use of intelligent data analysis techniques for system-level design: a software estimation example

Selection of Optimal Computing Platforms through the Suitability Measure

Contact Info

Product

Resources

About