RegHD: Robust and Efficient Regression in Hyper-Dimensional Learning System

Hernández-Cano, Alejandro; Zhuo, Cheng; Yin, Xunzhao; Imani, Mohsen

doi:10.1109/dac18074.2021.9586284

Cited by 19 publications

(13 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For example, previous HDC work (Hernández-Cano et al, 2021) uses Normal distribution N(0, 1) for p(ω) since it wants to approximate the Gaussian RBF kernel.…”

Section: Hyperdimensional Encoding In Vfamentioning

confidence: 99%

“…Our experimental results show that FLASH is about 5.5× faster in inference than RFF-based ridge regression while providing comparable or better accuracy. We also test a variant called "Accurate FLASH" that is optimized for accuracy, and this approach consistently outperforms other ML baselines, including the previous state-of-the-art HDC regression algorithm (Hernández-Cano et al, 2021) based on VFA. At the same time, we observe a linear increase in our approach with respect to the number of samples in the dataset, making this proposal particularly well-suited for large-scale data.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Hyperdimensional computing with holographic and adaptive encoder

Hernández-Cano,

Ni,

Zou

et al. 2024

Front. Artif. Intell.

Self Cite

View full text Add to dashboard Cite

IntroductionBrain-inspired computing has become an emerging field, where a growing number of works focus on developing algorithms that bring machine learning closer to human brains at the functional level. As one of the promising directions, Hyperdimensional Computing (HDC) is centered around the idea of having holographic and high-dimensional representation as the neural activities in our brains. Such representation is the fundamental enabler for the efficiency and robustness of HDC. However, existing HDC-based algorithms suffer from limitations within the encoder. To some extent, they all rely on manually selected encoders, meaning that the resulting representation is never adapted to the tasks at hand.MethodsIn this paper, we propose FLASH, a novel hyperdimensional learning method that incorporates an adaptive and learnable encoder design, aiming at better overall learning performance while maintaining good properties of HDC representation. Current HDC encoders leverage Random Fourier Features (RFF) for kernel correspondence and enable locality-preserving encoding. We propose to learn the encoder matrix distribution via gradient descent and effectively adapt the kernel for a more suitable HDC encoding.ResultsOur experiments on various regression datasets show that tuning the HDC encoder can significantly boost the accuracy, surpassing the current HDC-based algorithm and providing faster inference than other baselines, including RFF-based kernel ridge regression.DiscussionThe results indicate the importance of an adaptive encoder and customized high-dimensional representation in HDC.

show abstract

“…For example, previous HDC work (Hernández-Cano et al, 2021) uses Normal distribution N(0, 1) for p(ω) since it wants to approximate the Gaussian RBF kernel.…”

Section: Hyperdimensional Encoding In Vfamentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Hyperdimensional computing with holographic and adaptive encoder

Hernández-Cano,

Ni,

Zou

et al. 2024

Front. Artif. Intell.

Self Cite

View full text Add to dashboard Cite

show abstract

“…• RegHD; This approach [62] performs regression using data points that have been mapped in the hyperspace, thanks to the non-linearity of the HD/VSA encoding, this model is able to learn well and efficiently. This work uses the model to either perform regression or cluster data.…”

Section: Regressionmentioning

confidence: 99%

Classification using Hyperdimensional Computing: A~Review~with~Comparative Analysis

Vergés,

Heddes,

Nunes

et al. 2023

Preprint

View full text Add to dashboard Cite

Hyperdimensional computing (HD), also known as vector symbolic architectures (VSA), is an emerging and promising paradigm for cognitive computing. At its core, HD/VSA is characterized by its distinctive approach to representing information using high-dimensional random vectors. The recent surge in research within this field gains momentum from its remarkable computational efficiency and ability to excel in few-shot learning scenarios. Nonetheless, the current literature is deficient in providing a comprehensive comparative analysis of various methodologies since each method uses a different benchmark to evaluate its performance. This gap obstructs the monitoring of the field's state-of-the-art advancements and acts as a significant barrier to its overall progress. To address this gap, this review not only offers a conceptual overview of the latest literature but also introduces a comprehensive comparative study of HD/VSA classification techniques. Our exploration starts with an overview of the strategies proposed to encode information as high-dimensional vectors. These vectors serve as integral components in the construction of classification models. Furthermore, we evaluate diverse classification methods as proposed in the existing literature. This evaluation encompasses techniques such as retraining and regenerative training to augment the model's performance. To conclude our study, we present a comprehensive empirical study. This study serves as an in-depth analysis, systematically comparing various HD/VSA classification approaches using two benchmarks, the first being a set of seven popular datasets used in HD/VSA and the second consisting of 121 datasets extracted from the UCI Machine Learning repository.To facilitate future research on classification with HD/VSA, we open-sourced the benchmarking and the implementations of the methods we review. Our findings yield significant insights. Firstly, encodings based on key-value pairs emerge as optimal choices, boasting superior accuracy while maintaining high efficiency. Secondly, iterative adaptive methods demonstrate remarkable efficacy, potentially complemented by a regenerative strategy depending on the specific problem. Moreover, we show how HD/VSA is able to generalize while training with a limited number of training instances. Lastly, we demonstrate the robustness of HD/VSA methodologies by subjecting the model memory to a large number of bit-flips. Our results illustrate that the model's performance remains reasonably stable until the occurrence of 40\% of bit flips, where the model's performance is drastically degraded. Overall, we have performed a thorough performance evaluation on different methods and have observed a positive trend in both evaluation and execution time.

show abstract

“…In the past few years, HDC has gained significant traction as an emerging computing paradigm, especially for its deployments in machine learning and reasoning tasks. Prior works have proposed HDC-based algorithms and learning frameworks for classification [43], [4], [44], clustering [45], [46], regression [47], [9], and reinforcement learning [14], [48], [49] problems, showing the benefit of fast convergence in learning, high power/energy efficiency, natural data reuse and acceleration on customized devices [12], [11], [50], [51], and robustness on error-prone emerging hardware [52], [53]. Particularly, HDC has been successfully applied to many supervised learning tasks.…”

Section: Brain-inspired Hdcmentioning

confidence: 99%