RedMulE: A Compact FP16 Matrix-Multiplication Accelerator for Adaptive Deep Learning on RISC-V-Based Ultra-Low-Power SoCs

Abstract

The fast proliferation of extreme-edge applications using Deep Learning (DL) based algorithms required dedicated hardware to satisfy extreme-edge applications' latency, throughput, and precision requirements. While inference is achievable in practical cases, online finetuning and adaptation of general DL models are still highly challenging. One of the key stumbling stones is the need for parallel floating-point operations, which are considered unaffordable on sub-100 mW extreme-edge SoCs. We tackle this problem with RedMulE (Reduced-precision matrix Multiplication Engine), a parametric low-power hardware accelerator for FP16 matrix multiplications -the main kernel of DL training and inference -conceived for tight integration within a cluster of tiny RISC-V cores based on the PULP (Parallel Ultra-Low-Power) architecture. In 22 nm technology, a 32-FMA RedMulE instance occupies just 0.07 mm 2 (14% of an 8-core RISC-V cluster) and achieves up to 666 MHz maximum operating frequency, for a throughput of 31.6 MAC/cycle (98.8% utilization). We reach a cluster-level power consumption of 43.5 mW and a full-cluster energy efficiency of 688 16-bit GFLOPS/W. Overall, RedMulE features up to 4.65× higher energy efficiency and 22× speedup over SW execution on 8 RISC-V cores.

RISC-V Processor Technologies for Aerospace Applications in the ISOLDE Project

“…2). TPEs focus on accelerating matrix multiplication of the kind D = A × B + C, exploiting an internal high-efficiency systolic structure extending RedMulE [18,33], an open-source systolic array with multi-precision Fused Multiply-Add Modules that achieves up to 920 GFLOPS/W when operating on FP8 inputs with FP16 accumulators and 775 GFLOPS/W on full FP16. ISOLDE aims at further extending the TPE capabilities in several directions: more internal and input/output formats; tight integration with the RISC-V CVA6 cores to enable TPE utilization within performance-critical software code; larger performance gains; and better integration with software.…”

PULP Fiction No More—Dependable PULP Systems for Space

“…Figure 2 shows that a PULP cluster can be enhanced by introducing application-specific hardware accelerators that provide higher efficiency and performance during specific kernel execution over general-purpose cores. Since intense matrix multiplications are widespread in machine learning and deep learning, we integrate a Reduced-Precision Matrix Multiplication Engine, RedMulE [40], to introduce up to 22× better performance and 5× better energy efficiency on the execution of 16-bits floating-point matrix multiplication kernels over the parallel execution on the general-purpose RISC-V cores. Furthermore, we enhanced RedMulE with fault-tolerant capabilities to tackle safety-critical satellite onboard computing.…”

DARKSIDE: A Heterogeneous RISC-V Compute Cluster for Extreme-Edge On-Chip DNN Inference and Training

“…The Tensor Product Engine (TPE) [29] accelerates matrix multiplications (MatMuls) of the kind Z = X • W . It is designed to use the IEEE 754 binary-16 representation (FP16 in the following) since it is understood that FP16 can be used to train Neural Networks without significant accuracy loss, but reducing the power consumption and time to computation [30].…”