Survey and Benchmarking of Machine Learning Accelerators

Abstract

Advances in multicore processors and accelerators have opened the flood gates to greater exploration and application of machine learning techniques to a variety of applications. These advances, along with breakdowns of several trends including Moore's Law, have prompted an explosion of processors and accelerators that promise even greater computational and machine learning capabilities. These processors and accelerators are coming in many forms, from CPUs and GPUs to ASICs, FPGAs, and dataflow accelerators.This paper surveys the current state of these processors and accelerators that have been publicly announced with performance and power consumption numbers. The performance and power values are plotted on a scatter graph and a number of dimensions and observations from the trends on this plot are discussed and analyzed. For instance, there are interesting trends in the plot regarding power consumption, numerical precision, and inference versus training. We then select and benchmark two commerciallyavailable low size, weight, and power (SWaP) accelerators as these processors are the most interesting for embedded and mobile machine learning inference applications that are most applicable to the DoD and other SWaP constrained users. We determine how they actually perform with real-world images and neural network models, compare those results to the reported performance and power consumption values and evaluate them against an Intel CPU that is used in some embedded applications.On the left side of Figure 1, structured and unstructured data sources provide different views of entities and/or phenomenology. These raw data products are fed into a data conditioning step in which they are fused, aggregated, structured, accumulated, and converted to information. The information generated by the data conditioning step feeds into a host of supervised and unsupervised algorithms such as neural networks, which extract patterns, predict new events, fill in missing data, or look for similarities across datasets, thereby converting the input information to actionable knowledge. This actionable knowledge is then passed to human beings for decision-making processes in the human-machine teaming phase. The phase of human-machine teaming provides the users with useful and relevant insight turning knowledge into actionable intelligence or insight.Underlying all of these phases is a bedrock of modern computing systems that is comprised of one or more heterogenous computing elements. For example, sensor processing may occur on low power embedded computers, while algorithms may be computed in very large data centers. With regard to performance advances in these computing elements, Moore's law trends have ended [1], as have a number of related laws and trends including Denard's scaling (power density), clock frequency, core counts, instructions per clock cycle, and instructions per Joule (Koomey's law) [2]. Many of the technologies, tricks and techniques of processor chip designers that

Hardware Implementation of Deep Network Accelerators Towards Healthcare and Biomedical Applications

“…Table II shows a number of these CMOS-driven chips, which are intended for portable applications. There are many other examples of AI accelerator chips (for a comprehensive survey see [51]), but here we picked several prolific examples, which are designed specifically for DL using DNNs, RNNs, or both. We have also included a few general purpose AI accelerators from Google [52], Intel [53], and Huawei [54].…”

A Survey of Machine Learning Applications to Handover Management in 5G and Beyond

“…However, HO optimization often require real-time training and decision making. Hence, there is a need to reduce the number of parameters that needs to be trained by employing clustering method [126] and the use of hardware acceleration [186] to facilitate the ML training process. There might also be a need for both offline and online learning where the model goes through a periodic update and refinement during real time implementation.…”

Geospatial Artificial Intelligence: Potentials of Machine Learning for 3D Point Clouds and Geospatial Digital Twins

“…NVidia DGX/HGX; TensorFlow (Abadi et al 2016)]. They are known as AI accelerators, which differ with respect to hardware costs, training performance, computing power, and energy consumption (Wang et al 2019) but foster and simplify AI-based software development and operation (Reuther et al 2019).…”