SAFMem: Accelerating Transformer Self-Attention Functionality via Memristor-Based Hardware
Meriem Bettayeb,
Yasmin Halawani,
Muhammad Umair Khan
et al.
Abstract:The adoption of transformer networks has experienced a notable surge in various AI applications. However, the increasedcomputational complexity, stemming primarily from the self-attention mechanism, parallels the manner in which convolutionoperations constrain the capabilities and speed of Convolutional Neural Networks (CNNs). The self-attention algorithm,specifically the Matrix-matrix Multiplication (MatMul) operations, demands a substantial amount of memory and computationalcomplexity, thereby restricting th… Show more
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.