Models of parallel computation: a survey and classification

Abstract

In this paper, the state-of-the-art parallel computational model research is reviewed. We will introduce various models that were developed during the past decades. According to their targeting architecture features, especially memory organization, we classify these parallel computational models into three generations. These models and their characteristics are discussed based on three generations classification. We believe that with the ever increasing speed gap between the CPU and memory systems, incorporating non-uniform memory hierarchy into computational models will become unavoidable. With the emergence of multi-core CPUs, the parallelism hierarchy of current computing platforms becomes more and more complicated. Describing this complicated parallelism hierarchy in future computational models becomes more and more important. A semi-automatic toolkit that can extract model parameters and their values on real computers can reduce the model analysis complexity, thus allowing more complicated models with more parameters to be adopted. Hierarchical memory and hierarchical parallelism will be two very important features that should be considered in future model design and research.Keywords parallel computational models, hierarchical memory, hierarchical parallelism, three generations, memory model BackgroundThe simplified and abstract description of a computer is called a "computational model". A computer architect, algorithm designer and program developer can use such a model as a basis to assess their work including the suitability of one computer architecture to various applications, the computation complexity of an algorithm and the potential performance of one program on various computers, etc. A good computational model can simplify the complicated work of the architect, algorithm designer and program developer while mapping their work effectively onto real computers. Thus, such computational model is sometimes also called "Bridging model" [1]. The bridging model between the sequential computer and algorithm designer/program developer is the Von Neumann and RAM (Random Access Machine) Model [2]. However, no commonly recognized bridging models are found between parallel computer and parallel programs, and no other model exists that can map a user's parallel program so smoothly onto parallel computers as the Von Neumann and RAM Model do. This situation is largely due to the immature parallel computer design, i.e., there are so many different architectures for parallel computers that change rapidly each year, and the greater demand on performance [3]; a clean and simplified description is almost impossible. However, the trend of parallel computer design is converging and a common parallel computer architecture model can be realized (such as cluster), and the communication (we have standard MPI interface) of parallel computing is not so interconnect network dependent, thus we have the BSP and LogP models [1,7].Based on the historical development of parallel computational models, we think they can be classified ...

Integrated research of parallel computing: Status and future

“…We usually define parameters of machines, computing and cost function in general model, and they are called the three elements of the calculation model [16] . Computing model is formed by abstracting the entity (computer hardware/software) for the purpose of calculating.…”

Basic research in computer science and software engineering at SKLCS

“…N Computational model RAM(h,k) [45][46][47] It is a computational model with considerations of instruction/thread level parallelism (k ways) and memory hierarchy (h levels) based on the memory access complexity concept [48] { . We argue that the complexity analysis of algorithms should include traditional computation complexity (time and space complexity) and new memory access complexity (to reflect the different memory access behavior on memory hierarchy of different implementations).…”

“…Thus, we model the computer as one RAM with different memory access cost on each level under different memory access patterns. N Parallel computational model DRAM(h,k) model [45][46][47] It is the parallel version of RAM(h,k) model. It consists of p independent RAM(h,k) processors, each with its own local memory.…”

Survey of Parallel Computation Models

“…Проектировщики новых компьютеров, разработчики алгоритмов и программисты могут использовать модель вычислений для оценки своей работы, включая соответствие архитектуры компьютера различным приложениям, вычислительную сложность алгоритма и прогнозируемое быстродействие программы на различных вычислительных системах. Хорошая модель вычислений облегчает работу разработчиков компьютеров, алгоритмов и программ тем, что позволяет адекватно отобразить выбираемое решение на реальные компьютеры [1]. Подобную модель вычислений в литературе называют «соединяющей моделью (bridging model)» [2].…”