The Decline of Computers As a General Purpose Technology: Why Deep Learning and the End of Moore’s Law are Fragmenting Computing

Thompson, Neil; Spanuth, Svenja

doi:10.2139/ssrn.3287769

Cited by 20 publications

(14 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In recent years, the use of graphics processing units (GPUs) to speed up scientific computations has become commonplace. This has been, in part, dictated by the slow-down in Moore's law, and the progress made in the GPU architecture development, which has resulted in more computing capacity, flexibility, and ease-of-use [46]. In fact, currently a significant proportion of the world's fastest supercomputers is equipped with GPUs to accelerate their computations [42].…”

Section: Introductionmentioning

confidence: 99%

Implementing a GPU-based parallel MAX–MIN Ant System

Skinderowicz

2020

Future Generation Computer Systems

View full text Add to dashboard Cite

The MAX-MIN Ant System (MMAS) is one of the best-known Ant Colony Optimization (ACO) algorithms proven to be efficient at finding satisfactory solutions to many difficult combinatorial optimization problems. The slow-down in Moore's law, and the availability of graphics processing units (GPUs) capable of conducting general-purpose computations at high speed, has sparked considerable research efforts into the development of GPU-based ACO implementations. In this paper, we discuss a range of novel ideas for improving the GPU-based parallel MMAS implementation, allowing it to better utilize the computing power offered by two subsequent Nvidia GPU architectures. Specifically, based on the weighted reservoir sampling algorithm we propose a novel parallel implementation of the node selection procedure, which is at the heart of the MMAS and other ACO algorithms. We also present a memory-efficient implementation of another keycomponent -the tabu list structure -which is used in the ACO's solution construction stage. The proposed implementations, combined with the existing approaches, lead to a total of six MMAS variants, which are evaluated on a set of Traveling Salesman Problem (TSP) instances ranging from 198 to 3,795 cities. The results show that our MMAS implementation is competitive with state-of-the-art GPU-based and multi-core CPU-based parallel ACO implementations: in fact, the times obtained for the Nvidia V100 Volta GPU were up to 7.18x and 21.79x smaller, respectively. The fastest of the proposed MMAS variants is able to generate over 1 million candidate solutions per second when solving a 1,002-city instance. Moreover, we show that, combined with the 2-opt local search heuristic, the proposed parallel MMAS finds high-quality solutions for the TSP instances with up to 18,512 nodes.

show abstract

Section: Introductionmentioning

confidence: 99%

Implementing a GPU-based parallel MAX–MIN Ant System

Skinderowicz

2020

Future Generation Computer Systems

View full text Add to dashboard Cite

show abstract

“…Arguments "pro" and "against" exists for both of them. For example, Thompson & Spanuth (2018) advocate for the first scenario by linking the future dynamics of chips production to the dual-inducement mechanism typical of General Purpose Technologies (GPTs) (Bresnahan & Trajtenberg, 1995). They develop a model of choice between universal and specialised processors based on relative speed up factor and identify a cut-off point from which the specialised processors become more appealing than the universal ones.…”

Section: Scenarios and Discussionmentioning

confidence: 99%

On the Basis of Brain: Neural-Network-Inspired Change in General Purpose Chips

Prytkova¹,

Vannuccini

2020

SSRN Journal

View full text Add to dashboard Cite

In this paper, we disentangle the changes that the rise of Artificial Intelligence Technologies (AITs) is inducing in the semiconductor industry. The prevailing von Neumann architecture at the core of the established "intensive" technological trajectory of chip production is currently challenged by the rising difficulty to improve product performance over a growing set of computation tasks. In particular, the challenge is exacerbated by the increasing success of Artificial Neural Networks (ANNs) in application to a set of tasks barely tractable for classical programs. The inefficiency of the von Neumann architecture in the execution of ANN-based solutions opens room for competition and pushes for an adequate response from hardware producers in the form of exploration of new chip architectures and designs. Based on an historical overview of the industry and on collected data, we identify three characteristics of a chip-(i) computing power, (ii) heterogeneity of computation, and (iii) energy efficiency-as focal points of demand interest and simultaneously as directions of product improvement for the semiconductor industry players and consolidate them into a technoeconomic trilemma. Pooling together the trilemma and an analysis of the economic forces at work, we construct a simple model formalising the mechanism of demand distribution in the semiconductor industry, stressing in particular the role of its supporting services, the software domain. We conclude deriving two possible scenarios for chip evolution: (i) the emergence of a new dominant design in the form of a "platform chip" comprising heterogeneous cores; (ii) the fragmentation of the semiconductor industry into submarkets with dedicated chips. The convergence toward one of the proposed scenarios is conditional on (i) technological progress along the trilemma's edges, (ii) advances in the software domain and its compatibility with hardware, (iii) the amount of tasks successfully addressed by this software, (iv) market structure and dynamics.

show abstract

“…The third option of deeper co-design is less of a technological option than it is a new economic model for interacting with the industry that produces computer systems and the potential customers of said technologies. The era of general-purpose computing led to a more or less hands-off relationship between technology suppliers and their customers, as documented by Thompson & Spanuth [6], where a general-purpose processor could serve many different applications. In an era where specializing hardware to the application is the only means of performance improvement, the economic model for the design of future systems is going to need to change dramatically to lower design and verification costs for the development of new hardware.…”

Section: Architectural Specializationmentioning

confidence: 99%

The future of computing beyond Moore’s Law

Shalf

2020

Phil. Trans. R. Soc. A.

304

140

View full text Add to dashboard Cite

Moore’s Law is a techno-economic model that has enabled the information technology industry to double the performance and functionality of digital electronics roughly every 2 years within a fixed cost, power and area. Advances in silicon lithography have enabled this exponential miniaturization of electronics, but, as transistors reach atomic scale and fabrication costs continue to rise, the classical technological driver that has underpinned Moore’s Law for 50 years is failing and is anticipated to flatten by 2025. This article provides an updated view of what a post-exascale system will look like and the challenges ahead, based on our most recent understanding of technology roadmaps. It also discusses the tapering of historical improvements, and how it affects options available to continue scaling of successors to the first exascale machine. Lastly, this article covers the many different opportunities and strategies available to continue computing performance improvements in the absence of historical technology drivers. This article is part of a discussion meeting issue ‘Numerical algorithms for high-performance computational science’.

show abstract

The Decline of Computers As a General Purpose Technology: Why Deep Learning and the End of Moore’s Law are Fragmenting Computing

Cited by 20 publications

References 18 publications

Implementing a GPU-based parallel MAX–MIN Ant System

Implementing a GPU-based parallel MAX–MIN Ant System

On the Basis of Brain: Neural-Network-Inspired Change in General Purpose Chips

The future of computing beyond Moore’s Law

Contact Info

Product

Resources

About