Analysis of Memory Capacity for Deep Echo State Networks

Liu, Xuanlin; Chen, Mingzhe; Yin, Changchuan; Saad, Walid

doi:10.1109/icmla.2018.00072

Cited by 7 publications

(4 citation statements)

References 14 publications

(20 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…DeepESN requires a large reservoir size (U) to adequately train the randomly initialized weights of its hidden layers. The network performance with different reservoir configurations has been investigated based on datasets from both simulations and practical applications [48,49]. The general conclusion is that a higher U improves the prediction performance of DeepESN, while an excessively large U may lead to substantial computational resource consumption and overfitting.…”

Section: Deep Echo State Network (Deepesn) Performance Studymentioning

confidence: 99%

“…[47] reported a trade-off between the size of the reservoir and/or the number of training samples, due to the memory limit. Inheriting from ESNs, a DeepESN can achieve superior performance with an optimally configured reservoir, as discussed in [48][49][50][51]. DeepESNs with different reservoir configurations have been applied to several datasets, including nonlinear autoregressive moving average (NAMA), and for predictions pertaining to users' locations, orientations, and base station associations, respectively [49].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A Deep Echo State Network-Based Novel Signal Processing Approach for Underwater Wireless Optical Communication System with PAM and OFDM Signals

et al. 2023

View full text Add to dashboard Cite

Underwater wireless optical communication (UWOC) plays key role in the underwater wireless sensor networks (UWSNs), which have been widely employed for both scientific and commercial applications. UWOC offers high transmission data rates, high security, and low latency communication between nodes in UWSNs. However, significant absorption and scattering loss in underwater channels, due to ocean water conditions, can introduce highly non-linear distortion in the received signals, which can severely deteriorate communication quality. Consequently, addressing the challenge of processing UWOC signals with low optical signal-to-noise ratios (OSNRs) is critical for UWOC systems. Increasing the transmitting optical power and investigating more advanced signal processing technologies to recover transmitted symbols are two primary approaches to improve system tolerance in noisy UWOC signal channels. In this paper, we propose and demonstrate the application of deep echo state networks (DeepESNs) for channel equalization in high-speed UWOC systems to enhance system performance with both PAM and QPSK-OFDM modulations. Our experimental results demonstrate the effectiveness of DeepESNs in UWOC systems, achieving error-free underwater transmission over 40.5 m with data rates up to 167 Mbps. Moreover, we compare the performance of DeepESNs to conventional echo state networks and provide suggestions on the configuration of a DeepESN for UWOC signals.

show abstract

Section: Deep Echo State Network (Deepesn) Performance Studymentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

A Deep Echo State Network-Based Novel Signal Processing Approach for Underwater Wireless Optical Communication System with PAM and OFDM Signals

et al. 2023

View full text Add to dashboard Cite

show abstract

“…Next, we derive closed-form expressions of the memory capacity of the three ESN models that we described in Section III, namely, the single ESN model, the parallel ESN model, and the series ESN model. Note that, our previous work [35] analyzed the memory capacity for a centralized parallel ESN model. In contrast, here, we analyze the memory capacity for three ESN models used for federated learning.…”

Section: Memory Capacity Analysismentioning

confidence: 99%

Federated Echo State Learning for Minimizing Breaks in Presence in Wireless Virtual Reality Networks

Chen

Semiari

Saad

et al. 2020

IEEE Trans. Wireless Commun.

Self Cite

102

View full text Add to dashboard Cite

In this paper, the problem of enhancing the virtual reality (VR) experience for wireless users is investigated by minimizing the occurrence of breaks in presence (BIP) that can detach the users from their virtual world. To measure the BIP for wireless VR users, a novel model that jointly considers the VR application type, transmission delay, VR video quality, and users' awareness of the virtual environment is proposed. In the developed model, the base stations (BSs) transmit VR videos to the wireless VR users using directional transmission links so as to provide high data rates for the VR users, thus, reducing the number of BIP for each user. Since the body movements of a VR user may result in a blockage of its wireless link, the location and orientation of VR users must also be considered when minimizing BIP. The BIP minimization problem is formulated as an optimization problem which jointly considers the predictions of users' locations, orientations, and their BS association. To predict the orientation and locations of VR users, a distributed learning algorithm based on the machine learning framework of deep (ESNs) is proposed. The proposed algorithm uses concept from federated learning to enable multiple BSs to locally train their deep ESNs using their collected data and cooperatively build a learning model to predict the entire users' locations and orientations. Using these predictions, the user association policy that minimizes BIP is derived. Simulation results demonstrate that the developed algorithm reduces the users' BIP by up to 16% and 26%, respectively, compared to centralized ESN and deep learning algorithms.

show abstract

“…Another recent related model is the deep RC. Taking ESN as an example, the typical architecture of deep ESN can be classified into the series ESN and the parallel ESN, deriving other deep structures [13,30,61].…”

Section: Related Workmentioning

confidence: 99%

ViR:the Vision Reservoir

Wang¹,

Wang²,

Chen³

et al. 2021

Preprint

View full text Add to dashboard Cite

The most recent year has witnessed the success of applying the Vision Transformer (ViT) for image classification. However, there are still evidences indicating that ViT often suffers following two aspects, i) the high computation and the memory burden from applying the multiple Transformer layers for pre-training on a large-scale dataset, ii) the overfitting when training on small datasets from scratch. To address these problems, a novel method, namely, Vision Reservoir computing (ViR), is proposed here for image classification, as a parallel to ViT. By splitting each image into a sequence of tokens with a fixed length, the ViR constructs a pure reservoir with a nearly fully connected topology to replace the Transformer module in ViT. Two kinds of deep ViR models are subsequently proposed to enhance the network performance. Comparative experiments between the ViR and the ViT are carried out on several image classification benchmarks. Without any pre-training process, the ViR outperforms the ViT in terms of both model and computational complexity. Specifically, the number of parameters of the ViR is about 15% even 5% of the ViT, and the memory footprint is about 20% ∼ 40% of the ViT. The superiority of the ViR performance is explained by Small-World characteristics, Lyapunov exponents, and memory capacity.

show abstract

Analysis of Memory Capacity for Deep Echo State Networks

Cited by 7 publications

References 14 publications

A Deep Echo State Network-Based Novel Signal Processing Approach for Underwater Wireless Optical Communication System with PAM and OFDM Signals

A Deep Echo State Network-Based Novel Signal Processing Approach for Underwater Wireless Optical Communication System with PAM and OFDM Signals

Federated Echo State Learning for Minimizing Breaks in Presence in Wireless Virtual Reality Networks

ViR:the Vision Reservoir

Contact Info

Product

Resources

About