Kraken: A Direct Event/Frame-Based Multi-sensor Fusion SoC for Ultra-Efficient Visual Processing in Nano-UAVs

Mauro, Alfio Di; Scherer, Moritz; Rossi, Davide; Benini, Luca

doi:10.1109/hcs55958.2022.9895621

Cited by 14 publications

(17 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To offer enhanced computational capabilities within a small power budget, recent works also propose SoCs featuring hardwired ASIC accelerators designed for specific UAV applications, like, for example, motion-control [9], visual-inertial odometry (VIO) [10], simultaneous-localization-and-mapping (SLAM) [40], or QNN inference [7], [8]. These accelerators achieve impressive energy efficiency, in the order of hundreds of TOp/s/W, by carefully mapping the target algorithm to the hardware.…”

Section: B Socs For Nano-uavsmentioning

confidence: 99%

“…Exploiting both parallelism and reduced-precision computation is also a well-established technique to accelerate QNN inference and training, due to the nature of such algorithm. For example, accelerators like the NE16 in GAP9 [7], the HWCE in GAP8 [7], and the ternary weight neural-network (so-called CUTIE) accelerator in Kraken [8], able to reach peaks of 11.6 TMAC/s, have been proposed to speed up QNN inference. However, due to poor flexibility and programmability, these accelerators have to anyway rely on general-purpose CPUs to achieve end-to-end flight.…”

Section: B Socs For Nano-uavsmentioning

confidence: 99%

“…To overcome these limitations, recent MCUs integrate parallel fully-programmable and flexible accelerators, that have successfully proved to enable autonomous navigation [2], [3]. Namely, GAP8, GAP9 [7] and Kraken [8] are MCUs with enhanced computational capabilities, based on parallel programmable accelerators. GAP8 and GAP9 are commercial products by GreenWaves Technologies compliant with the socalled Crazyflie-AIdeck [41] board, which is meant as a companion of the Crazyflie to offload the mission control tasks [2].…”

Section: B Socs For Nano-uavsmentioning

confidence: 99%

“…Lastly, Kraken [8] is a research prototype based on the same heterogeneous architecture of GAP8 and GAP9, i.e., an RV32 CPU along with an eight RV32 core cluster, which delivers up to 90 GOp/s on 2-bit data. Kraken's RV32 cores are a more advanced version of Ri5cy, i.e., the Ri5cyNN cores [8] with support of sub-byte SIMD operations and fused Mac&Load instructions, which enable the concurrent execution of SIMD dot-product and memory accesses, increasing the computation efficiency up to 94%. Kraken embeds 1.5MB of on-chip SRAM memory and the CUTIE accelerator, able to achieve up to roughly 90k Ternary-MACs per cycle.…”

Section: B Socs For Nano-uavsmentioning

confidence: 99%

“…Usually, ultra-low-power (ULP) devices' accelerators are hardwired application-specific data-paths [9], [10] which achieve the best energy efficiency but are tailored to a single application domain, leading to poor programmability and a high nonrecurring engineering cost [12] while occupying a considerable part of the scarce area resources. To improve the overall versatility of the SoC, recent works replace ASIC accelerators with fully programmable parallel accelerators [7], [8] that achieve competitive energy efficiency while maintaining significant flexibility, and hence make the most out of the available power and area.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Shaheen: An Open, Secure, and Scalable RV64 SoC for Autonomous Nano-UAVs

Valente,

Veeran,

Sinigaglia

et al. 2023

2023 IEEE Hot Chips 35 Symposium (HCS)

View full text Add to dashboard Cite

The rapid advancement of energy-efficient parallel ultra-low-power (ULP) µcontrollers units (MCUs) is enabling the development of autonomous nano-sized unmanned aerial vehicles (nano-UAVs). These sub-10cm drones represent the next generation of unobtrusive robotic helpers and ubiquitous smart sensors. However, nano-UAVs face significant power and payload constraints while requiring advanced computing capabilities akin to standard drones, including real-time Machine Learning (ML) performance and the safe co-existence of general-purpose and real-time OSs. Although some advanced parallel ULP MCUs offer the necessary ML computing capabilities within the prescribed power limits, they rely on small main memories (<1MB) and µcontroller-class CPUs with no virtualization or security features, and hence only support simple bare-metal runtimes. In this work, we present Shaheen, a 9mm 2 200mW SoC implemented in 22nm FDX technology. Differently from state-of-the-art MCUs, Shaheen integrates a Linux-capable RV64 core, compliant with the v1.0 ratified Hypervisor extension and equipped with timing channel protection, along with a low-cost and low-power memory controller exposing up to 512MB of off-chip low-cost low-power HyperRAM directly to the CPU. At the same time, it integrates a fully programmable energy-and area-efficient multi-core cluster of RV32 cores optimized for general-purpose DSP as well as reduced-and mixed-precision ML. To the best of the authors' knowledge, it is the first silicon prototype of a ULP SoC coupling the RV64 and RV32 cores in a heterogeneous host+accelerator architecture fully based on the RISC-V ISA. We demonstrate the capabilities of the proposed SoC on a wide range of benchmarks relevant to nano-UAV applications including general-purpose DSP as well as inference and online learning of quantized DNNs. The cluster can deliver up to 90GOp/s and up to 1.8TOp/s/W on 2-bit integer kernels and up to 7.9GFLOp/s and up to 150GFLOp/s/W on 16-bit FP kernels.

show abstract

Section: B Socs For Nano-uavsmentioning

confidence: 99%

Section: B Socs For Nano-uavsmentioning

confidence: 99%

Section: B Socs For Nano-uavsmentioning

confidence: 99%

Section: B Socs For Nano-uavsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Shaheen: An Open, Secure, and Scalable RV64 SoC for Autonomous Nano-UAVs

Valente,

Veeran,

Sinigaglia

et al. 2023

2023 IEEE Hot Chips 35 Symposium (HCS)

View full text Add to dashboard Cite

show abstract

Networks-on-Chip to Enable Large-Scale Multi-core ML Acceleration

Jain,

Verhelst

2023

Towards Heterogeneous Multi-Core Systems-on-Chip for Edge Machine Learning

View full text Add to dashboard Cite

Hybrid chips to enable a sustainable internet of things technology: opportunities and challenges

Rogdakis,

Psaltakis,

Fagas

et al. 2024

Discov Mater

View full text Add to dashboard Cite

A new technological approach is needed for the development of emerging electronic components and systems within the Internet of Things (IoT) era. New advancements and innovations are required in architectural design and hybrid systems heterogeneous integration to address the challenge of widespread IoT deployment, its power demands and sustainability. Hybrid chips are one of the emerging technologies that can help overcome the current limitations in terms of energy consumption, performance and sustainability that could shape the future of electronic systems for targeted applications. Hybrid chips combine different materials and manufacturing technologies on the same substrate or package using advanced flexible heterogeneous integration techniques, with the focus of merging the advantages of each unit or technology toward enhanced performance and new levels of emerging functionalities. The categorization of hybrid chips spans across rigid hybrid chips that follow a multi-chiplet approach, semi-flexible chips that integrate flexible units with semi-rigid ones such as thinned silicon integrated circuits, and lastly, flexible chips in which all components are inherently flexible. This perspective article analyzes technical challenges that arise concerning the development of sustainable materials and processing technologies, complex heterogeneous integration, as well as advanced packaging architectures, standardization, and reliability testing. The economic implications for the semiconductor technology transition to hybrid chips is presented in terms of manufacturing costs, economic feasibility, and market readiness. Key insights and future application opportunities are provided, while recent advancements in the field are summarized. This perspective article suggests that by addressing these challenges, hybrid chips have the potential to transform electronic components and systems across a wide range of industries and use case scenario. The advancement of hybrid chip technologies by enhancing the collaboration between industry and academia as well as policymakers will be a crucial part in the realization of required sustainability goals within the worldwide Chips Act initiative, while ensuring the technological progress has the right balance between improved performance and sustainability.

show abstract

Kraken: A Direct Event/Frame-Based Multi-sensor Fusion SoC for Ultra-Efficient Visual Processing in Nano-UAVs

Cited by 14 publications

References 7 publications

Shaheen: An Open, Secure, and Scalable RV64 SoC for Autonomous Nano-UAVs

Shaheen: An Open, Secure, and Scalable RV64 SoC for Autonomous Nano-UAVs

Networks-on-Chip to Enable Large-Scale Multi-core ML Acceleration

Hybrid chips to enable a sustainable internet of things technology: opportunities and challenges

Contact Info

Product

Resources

About