34.4 A 3nm, 32.5TOPS/W, 55.0TOPS/mm² and 3.78Mb/mm² Fully-Digital Compute-in-Memory Macro Supporting INT12 × INT12 with a Parallel-MAC Architecture and Foundry 6T-SRAM Bit Cell

Towards efficient generative AI and beyond-AI computing: New trends on ISSCC 2024 machine learning accelerators

“…They realize an ultra-low-power on-device inference system by hybrid multiplication/accumulation units, speculative decoding, and implicit weight generation, reducing external memory access (EMA) by 74%−81%. In addition, a special forum named Energy-Efficient AI-Computing Systems for Large-Language Models shares more practical thoughts about large language model (LLM) computing systems [9] . Georgia Institute of Technology, NVIDIA, Intel, Google, KAIST, Samsung, Axelera AI, and MediaTek introduce their latest research over LLM training and inference in both cloud and edge.…”