In-Hardware Training Chip Based on CMOS Invertible Logic for Machine Learning

Onizawa, Naoya; Smithson, Sean C.; Meyer, Brett H.; Gross, Warren J.; Hanyu, Takahiro

doi:10.1109/tcsi.2019.2960383

Cited by 22 publications

(23 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…One such promising approach to on-chip learning is using Boltzmann machines to implement invertible logic [90]. Invertible logic can perform operations in both the forward and reverse directions using the same hardware circuit.…”

Section: B Low-complexity Trainingmentioning

confidence: 99%

“…For example, an invertible multiplier (forward) and factorizer (backward) was implemented using a Boltzmann machine using a standard CMOS process [92]. Recent results have demonstrated a CMOS chip that implements neural inference (forward) and training (backward) in the same hardware with invertible logic [90]. In this case, it is possible to directly obtain the values of the weights with low-precision computations, enabling low-complexity on-chip learning.…”

Section: B Low-complexity Trainingmentioning

confidence: 99%

“…The global minima are obtained by clamping the outputs of any given nodes and allowing the remaining to be computed. In [90], the equation expressing the forward computations of a neuron is converted to Boltzmann machine structure:…”

Section: B Low-complexity Trainingmentioning

confidence: 99%

See 2 more Smart Citations

Hardware-Aware Design for Edge Intelligence

Gross

Meyer

Ardakani

2021

IEEE Open J. Circuits Syst.

Self Cite

View full text Add to dashboard Cite

Section: B Low-complexity Trainingmentioning

confidence: 99%

Section: B Low-complexity Trainingmentioning

confidence: 99%

See 1 more Smart Citation

Hardware-Aware Design for Edge Intelligence

Gross

Meyer

Ardakani

2021

IEEE Open J. Circuits Syst.

Self Cite

View full text Add to dashboard Cite

“…The bidirectional computing capability is realized by reducing the network energy to the global minimum energy with noise induced by random signals (e.g., a multiplier can be used as a factorizer in the backward mode). Due to the unique feature, several challenging problems can be quickly solved, such as integer factorization (e.g., cryptography problems [1]) and machine learning (e.g., training neural networks [3], [4]).…”

Section: Introductionmentioning

confidence: 99%

Sparse Random Signals for Fast Convergence on Invertible Logic

et al. 2021

Self Cite

View full text Add to dashboard Cite

This paper introduces sparse random signals for fast convergence on invertible logic. Invertible logic based on a network of probabilistic nodes (spins), similar to a Boltzmann machine, can compute functions bidirectionally by reducing the network energy to the global minimum with the addition of random signals. Here, we propose using sparse random signals that are generated by replacing a part of the typical dense random signals with zero values in probability. The sparsity of the random signals can induce a relatively relaxed transition of the spin network, reaching the global minimum energy at high probabilities. As a typical design example of invertible logic, invertible adders and multipliers are designed and evaluated. The simulation results show that the convergence speed with the proposed sparse random signals is roughly an order of magnitude faster than that with the conventional dense random signals. In addition, several key parameters are found and could be a guideline for fast convergence on general invertible logic.

show abstract

“…Recently, researchers have been engaged in the demanding work of building brainware computing system [7]- [10]. An in-hardware training chip has been fabricated and carried out a demonstration for data classification, which exhibits a noticeable reduction of power dissipation and latency [11]. Pei et al has proposed a hybrid Tianjic chip architecture that consists of multiple cores, reconfigurable building blocks to achieve the precise control and real-time object detection of unmanned bicycle [12].…”

Section: Introductionmentioning

confidence: 99%

A Systematic Study of Tiny YOLO3 Inference: Toward Compact Brainware Processor With Less Memory and Logic Gate

Endoh

2020

IEEE Access

View full text Add to dashboard Cite

The emerging of deep neural networks, especially the convolutional neural network (CNN), substantially promotes the fast development of brainware processors in object detection. However, the vast network architecture brings severe challenges to the design of brainware processor, which requires a large number of logic gates and memories. Therefore, a compact brainware processor with less memory and logic gate has a high demand in object detection. Typically, the object detection involves single-shot and multishot detectors in accordance with different detection principle. In the early stage, the multi-shot detector has a leading role in solving object detection issues, such as region-based convolutional neural networks (R-CNNs), faster R-CNNs etc. However, the multi-shot detector suffers from a low detection rate comparing with the single-shot detector. The you only look once (YOLO) algorithm, as the state-of-the-art real-time object detection algorithm, receives extensive attention from the academics and industry. Particularly, the lightweight YOLO algorithm, tiny YOLO3, has excellent potential for circuit design of compact brainware processor. Nonetheless, systematic studies of tiny YOLO3 are still missing up to the present. This paper offers a thorough review of the tiny YOLO3 algorithm, which can fill the gap in the field of object detection. Furthermore, the open solutions of compressing the tiny YOLO3 algorithm are proposed from the aspects of algorithm, hardware and emerging technology. The comprehensive study presented in this paper can not only enhance understanding of the tiny YOLO3 algorithm for researchers or engineers but also make a significant contribution to accelerating the development of compact brainware processor. INDEX TERMSTiny YOLO3, brainware processor, deep neural network, CNN, hardware acceleration VOLUME 4, 2016 II. RELATED WORK A. YOLO DEVELOPMENT J. Redmon firstly proposed the YOLO algorithm in May 2016 [22], and it has been evolved to four generations within four years. The base YOLO, motivated by fast R-CNNs, introduces the region-based concept to the neural network.Peculiarly, the input image is divided into different grid cells where two bounding boxes are predicted. In each bounding box, the center coordinate of object, confidence scores and the class probabilities are predicted. The confidence score is responsible for checking whether or not the object exists in the bounding box. The base YOLO has a good advantage in the perspective of speed and acceptable accuracy, but the following drawbacks retard the development of base YOLO:• The base YOLO is hard to handle the situation that the distance of two objects is very close. The detector may

show abstract

In-Hardware Training Chip Based on CMOS Invertible Logic for Machine Learning

Cited by 22 publications

References 31 publications

Hardware-Aware Design for Edge Intelligence

Hardware-Aware Design for Edge Intelligence

Sparse Random Signals for Fast Convergence on Invertible Logic

A Systematic Study of Tiny YOLO3 Inference: Toward Compact Brainware Processor With Less Memory and Logic Gate

Contact Info

Product

Resources

About