ASIC-Based Implementation of Random Spray Retinex Algorithm for Image Enhancement

Bettayeb, Meriem; Tesfai, Huruy; Mohammad, Baker; Saleh, Hani

doi:10.1109/mwscas54063.2022.9859352

Cited by 5 publications

(1 citation statement)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In addition to conventional solutions, the memristor crossbar architecture presents itself as a promising design choice owing to its extensive parallelism, high density, low access power, and the potential for multi-level cell and 3D integration, as evidenced by recent studies 19,20 . In the form of programmable conductance, a single memristor cell has the capacity to store binary or multi-bit information, providing a versatile platform for both analog and digital IMC operations, as discussed in the literature 21 .…”

Section: Introductionmentioning

confidence: 99%

SAFMem: Accelerating Transformer Self-Attention Functionality via Memristor-Based Hardware

Bettayeb,

Halawani,

Khan

et al. 2023

Preprint

Self Cite

View full text Add to dashboard Cite

The adoption of transformer networks has experienced a notable surge in various AI applications. However, the increasedcomputational complexity, stemming primarily from the self-attention mechanism, parallels the manner in which convolutionoperations constrain the capabilities and speed of Convolutional Neural Networks (CNNs). The self-attention algorithm,specifically the Matrix-matrix Multiplication (MatMul) operations, demands a substantial amount of memory and computationalcomplexity, thereby restricting the overall performance of the transformer. This paper introduces an efficient hardwareaccelerator for the transformer network, leveraging memristor-based in-memory computing. The design targets the memorybottleneck associated with MatMul operations in the self-attention process, utilizing approximate analog computation andthe highly parallel computations facilitated by the memristor crossbar architecture. Remarkably, this approach resulted in areduction of approximately 10 times in the number of Multiply-Accumulate (MAC) operations in transformer networks, whilemaintaining 93.37% accuracy for the MNIST dataset, as validated by a comprehensive circuit simulator employing NeuroSim3.0. Simulation outcomes indicate an area utilization of 6895.7 μm2, a latency of 15.52 seconds, an energy consumption of3 mJ, and a leakage power of 59.55 μW . The methodology outlined in this paper represents a substantial stride towards ahardware-friendly transformer architecture for edge device, poised to achieve real-time performance.

show abstract

Section: Introductionmentioning

confidence: 99%