Analog Coding in Emerging Memory Systems

Abstract

Exponential growth in data generation and large-scale data science has created an unprecedented need for inexpensive, low-power, low-latency, high-density information storage. This need has motivated significant research into multi-level memory devices that are capable of storing multiple bits of information per device. The memory state of these devices is intrinsically analog. Furthermore, much of the data they will store, along with the subsequent operations on the majority of this data, are all intrinsically analog-valued. Ironically though, in the current storage paradigm, both the devices and data are quantized for use with digital systems and digital error-correcting codes. Here, we recast the storage problem as a communication problem. This then allows us to use ideas from analog coding and show, using phase change memory as a prototypical multi-level storage technology, that analog-valued emerging memory devices can achieve higher capacities when paired with analog codes. Further, we show that storing analog signals directly through joint coding can achieve low distortion with reduced coding complexity. Specifically, by jointly optimizing for signal statistics, device statistics, and a distortion metric, we demonstrate that single-symbol analog codings can perform comparably to digital codings with asymptotically large code lengths. These results show that end-to-end analog memory systems have the potential to not only reach higher storage capacities than discrete systems but also to significantly lower coding complexity, leading to faster and more energy efficient data storage. A paradigm shift in the type and quantity of storable information is currently underway. Internet-connected devices are projected to reach 50 billion, more than 6 devices per person, by the year 2020 1. Much of the data produced by these devices, such as pixel intensities from cameras, sound recordings from microphones, and time-series from sensors, comes from signals that are intrinsically analog-valued or many-valued (>100 values) and ordinal. These signals are also highly redundant and compressible, with a large degree of their joint activity explained by a smaller number of factors than the intrinsic dimensionality of the signal. Furthermore, an increasing portion of the operations performed on the data are analog, with much of it being used for statistical inference or human perception (e.g. object-detection for images). A concurrent paradigm shift is underway in the media we use to store data. Early storage media such as phonograph records and VCR tapes relied on perturbing an analog-valued state (wax height and magnetic polarization, respectively). Digital computation led to the popularity of binary storage representations that inhibit noise propagation and utilize the concurrently developed theories of binary error-correcting codes 2. However, many emerging memory technologies have shifted back to analog-valued media to create multi-level devices that fill the need for inexpensive, high-density storage. MLC-Flash, Phase Ch...

Iron Oxide-Coupled Graphite/Fe–Si Steel Structure for Analog Computing from Recycling Principle

“…In addition, it is known that electric and thermal stresses during switching conduction of transistors cause limitations as coding faults when these are operating at higher speeds [12,19]. To improve reliability in digital systems, emergent schemes in recent years are generating renewed interest, whereby continuous growth of neuromorphic computing for bio-inspired data processing and hardware emulation of cellular building blocks are widely explored to perform analog-valued encoding [13]. However, a growing number of building blocks, difficulty to configure arbitrary connections between distant blocks of devices and scaling linearly with the size are some drawbacks in analog schemes.…”

“…Today, analog computing is proposed and studied extensively to be capable of executing easily arithmetic functions at much lower power consumption than digital solutions. An analog scheme can allow versatile conduction modes under a wide range of conditions, which means that continuous time processing will be more beneficial than discrete time processing [12,13]. Thus, seeking ecofriendly alternatives for analog computing within a reduced number of building blocks is of importance.…”

High-density analog image storage in an analog-valued non-volatile memory array

“…Our prior work related to the current work are [5,19,20], which considered the problem of storing analog signal with analog memory devices, but without large-scale hardware demonstration. To be specific, [5] characterized the channel capacity of a single analog PCM device and showed a way of storing a Gaussian source with a single-symbol analog coding; [19] disclosed the JSCC concept by simulating grey-scale image storage on idealized modeled PCM devices; [20] stored by experiment one gray-level image onto 1 K-cell analog RRAM array with JSCC scheme.…”

“…Our prior work related to the current work are [5,19,20], which considered the problem of storing analog signal with analog memory devices, but without large-scale hardware demonstration. To be specific, [5] characterized the channel capacity of a single analog PCM device and showed a way of storing a Gaussian source with a single-symbol analog coding; [19] disclosed the JSCC concept by simulating grey-scale image storage on idealized modeled PCM devices; [20] stored by experiment one gray-level image onto 1 K-cell analog RRAM array with JSCC scheme. The current work presents hardware demonstrations of 100 RGB images storage with JSCC scheme on both large scale (1 M-cell) PCM array and large scale (1 M-cell) RRAM array at different compression ratios, statistically analyses of JSCC experimental performance compared with conventional digital storage of images, and comprehensively investigates the impact of both PCM and RRAM device non-idealities on JSCC performance.…”

“…To achieve this goal, the first objective is to maximize the NVM data storage capacity. While the usual multi-bit-cell (MLC) storage is based on quantizing the resistance into multiple discrete levels, [5] has previously shown that storing analog codes directly into analog-valued NVM would improve the rate of information transmission over digital codes and at the same time lower coding complexity. In this work, we use analog-valued NVM to maximize the storage capacity of each memory device, and adaptive joint source-channel coding (JSCC) scheme to efficiently store the analog data in a compressed form into analog-valued NVM.…”