Machine Learning in Resource-Scarce Embedded Systems, FPGAs, and End-Devices: A Survey

Abstract

The number of devices connected to the Internet is increasing, exchanging large amounts of data, and turning the Internet into the 21st-century silk road for data. This road has taken machine learning to new areas of applications. However, machine learning models are not yet seen as complex systems that must run in powerful computers (i.e., Cloud). As technology, techniques, and algorithms advance, these models are implemented into more computational constrained devices. The following paper presents a study about the optimizations, algorithms, and platforms used to implement such models into the network’s end, where highly resource-scarce microcontroller units (MCUs) are found. The paper aims to provide guidelines, taxonomies, concepts, and future directions to help decentralize the network’s intelligence.

An Overview of Machine Learning within Embedded and Mobile Devices–Optimizations and Applications

“…As can be observed in this table, microcontrollers have very limited hardware resources. This scarcity of resources makes them unsuitable for high-end machine learning applications, except the machine learning models are heavily optimized to fit within this space [ 24 ].…”

“…However, graphic processing units (GPUs), due to their high floating-point performance and thread-level parallelism, are more suitable for training deep learning models [ 13 ]. Extensive research is actively being carried out to develop suitable hardware acceleration units using FPGAs [ 20 , 21 , 22 , 23 , 24 , 25 , 26 ], GPUs, ASICs, and TPUs to create heterogeneous and sometimes distributed systems to meet up the high computational demand of deep learning models. At both the algorithm and hardware levels, optimization techniques for classical machine learning and deep learning algorithms are being investigated such as pruning, quantization, reduced precision, hardware acceleration, etc.…”

“…They, however, only explored a type of deep learning model (DNNs) and did not discuss other deep learning models (CNN, RNN), which have gained attention in recent times. Sergio et al [ 24 ] have carried out a comprehensive survey on ML in embedded and mobile devices, presenting ML concepts and techniques for optimization and also investigated different application areas. They, however, also do not explore other models of DNNs or make appropriate trade-offs.…”

Improving the Characteristics of Multi-Level LUT-Based Mealy FSMs

“…Using (11) gives R Q = 3 and the set Z = {z 1 , z 2 , z 3 }. Using (15) gives R I = 1 and the set V = {v 1 }.…”

“…In this article we consider methods of implementing FSM circuits in the context of field programmable gate arrays [11][12][13]. These chips are very popular devices used for implementations of digital systems [2,[14][15][16][17][18]. This fact explains our choice of FPGA-based Mealy FSMs as a research object.…”