No abstract

In this paper, we revisit the question of how ML algorithms can be best integrated into existing DBMSs to not only avoid expensive data copies to external ML tools but also to comply with regulatory reasons. The key observation is that database transactions already provide an execution model that allows DBMSs to efficiently mimic the execution model of modern parallel ML algorithms. As a main contribution, this paper presents DB4ML, an in-memory database kernel that allows applications to implement user-defined ML algorithms and efficiently run them inside a DBMS. Thereby, the ML algorithms are implemented using a programming model based on the idea of so called iterative transactions. Our experimental evaluation shows that DB4ML can support user-defined ML algorithms inside a DBMS with the efficiency of modern specialized ML engines. In contrast to DB4ML, these engines not only need to transfer data out of the DBMS but also hard-code the ML algorithms and thus are not extensible.

No abstract

Binary Neural Networks In article number http://doi.wiley.com/10.1002/aisy.202000134, Stephan Menzel and co‐workers explore a computation in‐memory concept for binary vector‐matrix multiplications based on complementary resistive switches. Experimental results on a small‐scale demonstrator are shown and the influence of device variability is investigated. The simulated inference of a 1‐layer fully connected binary neural network trained on the MNIST data set resulted in an accuracy of nearly 86%.

With artificial neural networks (ANNs) becoming more and more powerful and with the slowdown of complementary metal-oxide-semiconductor (CMOS) scaling, the Von Neumann memory wall is becoming an increasingly prominent problem for ANN hardware systems. [1,2] Large neural networks especially suffer from this because not all computational information necessary can be stored in the cache memory, and costly communication with higher level storage is necessary. [3] In software, techniques such as pruning, weight reuse, or reducing the quantization are used to create less complex but still accurate ANNs. [4] A promising simplification from the hardware perspective is the aforementioned reduction in quantization due to its reduced computational complexity. The number of quantization levels can even go down to the bare minimum of two levels and results in binary neural networks (bNNs) which have been heavily studied in recent years. [5-8] These networks use the values 1 and À1 to encode their weights and activations during the inference step making them the most efficient ANNs possible to compute in hardware. [9] With ongoing improvements in prediction accuracy and the development of new hardware accelerators, these networks are auspicious candidates for computing ANNs on edge devices. Apart from the use of CMOS accelerators, new emerging technologies based on resistive switching devices can play a crucial role in this advancement. [10] These devices can mimic the synaptic weights in ANNs and, configured in a crossbar architecture, enable fast analog computations of vector-matrix multiplications, which are the main operations in ANNs. [11,12] One promising class of resistive switching devices relies on redox reactions and is therefore called redox-based resistive switching devices, also known as redox-based random access memory (ReRAM). [13] They are typically based on metaloxide-metal stacks and can change the conduction through the oxide layer based on electric signals applied to the metal electrodes. These changes in resistance from a high resistive state (HRS) to a low resistance state (LRS) and vice versa originate from ionic movements in the oxide layer and concurrent redox reactions. [14] There are already many realizations of ReRAM crossbar arrays used as accelerators for vector-matrix multiplication. [15-17] One common type uses Kirchoff 's current law to do the computation where the calculation result is encoded in the resulting current. To compute this result, the current has to be sensed, which becomes more difficult the lower the current is. This leads to a trade-off in the circuit design as a lower current reduces the energy consumption. Another commonly used architecture builds up a resistive voltage divider between a sense resistance and the resistive crossbar array. In this architecture, the result of the computation is encoded in the voltage drop of the voltage divider. [18] One design challenge of this architecture is the

The proliferation of machine learning algorithms in everyday applications such as image recognition or language translation has increased the pressure to adapt underlying computing architectures towards these algorithms. Application specific integrated circuits (ASICs) such as the Tensor Processing Units by Google, Hanguang by Alibaba or Inferentia by Amazon Web Services were designed specifically for machine learning algorithms and have been able to outperform CPU based solutions by great margins during training and inference. As newer generations of chips allow handling of and computation on more and more data, the size of neural networks has dramatically increased, while the challenges they are trying to solve have become more complex. Neuromorphic computing tries to take inspiration from biological information processing systems, aiming to further improve the efficiency with which these networks can be trained or the inference can be performed. Enhancing neuromorphic computing architectures with memristive devices as non-volatile storage elements could potentially allow for even higher energy efficiencies. Their ability to mimic synaptic plasticity dynamics brings neuromorphic architectures closer to the biological role models. So far, memristive devices are mainly investigated for the emulation of the weights of neural networks during training and inference as their non-volatility would enable both processes in the same location without data transfer. In this paper, we explore realisations of different synapses build from memristive ReRAM devices, based on the Valence Change Mechanism. These synapses are the 1R synapse, the NR synapse and the 1T1R synapse. For the 1R synapse, we propose three dynamical regimes and explore their performance through different synapse criteria. For the NR synapse, we discuss how the same dynamical regimes can be addressed in a more reliable way. We also show experimental results measured on ZrOx devices to support our simulation based claims. For the 1T1R synapse, we explore the trade offs between the connection direction of the ReRAM device and the transistor. For all three synapse concepts we discuss the impact of device-to-device and cycle-to-cycle variability. Additionally, the impact of the stimulation mode on the observed behavior is discussed.

No abstract

No abstract