Stage-Wise Magnitude-Based Pruning for Recurrent Neural Networks

Li, Guiying; Yang, Peng; Qian, Chao; Hong, Richang; Tang, Ke

doi:10.1109/tnnls.2022.3184730

Cited by 4 publications

(3 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These networks capture temporal patterns in data, aiding in forecasts and financial analysis. RNNs face challenges like the vanishing gradient problem, addressed by advances in LSTM and GRU technologies, enhancing their capability to remember information over extended periods [34]. Despite computational demands, RNNs' unique architecture makes them invaluable for economic research and policymaking, offering insights into temporal economic dynamics.…”

Section: Recurrent Neural Networkmentioning

confidence: 99%

Deep learning for economic transformation: a parametric review

Tariq,

Ahmed,

Khan

et al. 2024

IJEECS

View full text Add to dashboard Cite

Deep learning (DL) is increasingly recognized for its effectiveness in analyzing and forecasting complex economic systems, particularly in the context of Pakistan's evolving economy. This paper investigates DL's transformative role in managing and interpreting increasing volumes of intricate economic data, leading to more nuanced insights. DL models show a marked improvement in predictive accuracy and depth over traditional methods across various economic domains and policymaking scenarios. Applications include demand forecasting, risk evaluation, market trend analysis, and resource allocation optimization. These processes utilize extensive datasets and advanced algorithms to identify patterns that traditional methods cannot detect. Nonetheless, DL's broader application in economic research faces challenges like limited data availability, complexity of economic interactions, interpretability of model outputs, and significant computational power requirements. The paper outlines strategies to overcome these barriers, such as enhancing model interpretability, employing federated learning for better data privacy, and integrating behavioral and social economic theories. It concludes by stressing the importance of targeted research and ethical considerations in maximizing DL's impact on economic insights and innovation, particularly in Pakistan and globally.

show abstract

Section: Recurrent Neural Networkmentioning

confidence: 99%

Deep learning for economic transformation: a parametric review

Tariq,

Ahmed,

Khan

et al. 2024

IJEECS

View full text Add to dashboard Cite

show abstract

“…The implicit assumption is that, for the SAERL framework, the fitness of candidate solutions are only affected by a few dimensions instead of all. This assumption is technically reasonable for the DNN-based policy since DNN often preserves redundent weights and can be effectively prunned [10,7,8].…”

Section: Pe Module : Random Embeddingmentioning

confidence: 99%

“…We vary the parameter value over a wide range and reevaluate to perform parameter sensitivity analysis. More specifically, we choose the number of candidate policies involved in pre-selection in the surrogate model as the hyperparameter, and set it to four different values as [3,5,10,100]. We also select three games (i.e., BeamRider, Bowling, Freeway) for sensitivity analysis.…”

Section: Sensitivity Analysismentioning

confidence: 99%

Enabling surrogate-assisted evolutionary reinforcement learning via policy embedding

Tang¹,

Li²,

Zhang³

et al. 2023

Preprint

View full text Add to dashboard Cite

Evolutionary Reinforcement Learning (ERL) that applying Evolutionary Algorithms (EAs) to optimize the weight parameters of Deep Neural Network (DNN) based policies has been widely regarded as an alternative to traditional reinforcement learning methods. However, the evaluation of the iteratively generated population usually requires a large amount of computational time and can be prohibitively expensive, which may potentially restrict the applicability of ERL. Surrogate is often used to reduce the computational burden of evaluation in EAs. Unfortunately, in ERL, each individual of policy usually represents millions of weights parameters of DNN. This high-dimensional representation of policy has introduced a great challenge to the application of surrogates into ERL to speed up training. This paper proposes a PE-SAERL Framework to at the first time enable surrogate-assisted evolutionary reinforcement learning via policy embedding (PE). Empirical results on 5 Atari games show that the proposed method can perform more efficiently than the four state-of-the-art algorithms. The training process is accelerated up to 7x on tested games, comparing to its counterpart without the surrogate and PE.

show abstract