Structured Pruning of LSTMs via Eigenanalysis and Geometric Median for Mobile Multimedia and Deep Learning Applications

Gkalelis, Nikolaos; Mezaris, Vasileios

doi:10.1109/ism.2020.00028

Cited by 4 publications

(2 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…removing network connections and/or nodes to reduce the complexity, e.g. [53,54]. Classification in the media domain is almost assuredly multi-label, i.e.…”

Section: Classification Of Media Assetsmentioning

confidence: 99%

Data-driven personalisation of television content: a survey

et al. 2022

Self Cite

View full text Add to dashboard Cite

show abstract

“…removing network connections and/or nodes to reduce the complexity, e.g. [53,54]. Classification in the media domain is almost assuredly multi-label, i.e.…”

Section: Classification Of Media Assetsmentioning

confidence: 99%

Data-driven personalisation of television content: a survey

et al. 2022

Self Cite

View full text Add to dashboard Cite

show abstract

“…Outra estratégia utilizada para compressão de redes é a utilização de Operadores de Produto de Matriz (MPO) para representação dos pesos sinápticos, o que reduz a quantidade de parâmetros armazenados (Gao et al, 2020). Há autores que também realizam pruning a partir análises de autovalores e médias geométricas dos pesos e eliminando as unidades mais redundantes da LSTM (Gkalelis and Mezaris, 2020).…”

Section: Introductionunclassified

Análise de Desempenho de Redes Neurais LSTM com Técnicas de Pruning para Detecção de Falhas em Processos Industrias

Correia

Dantasy

Guedes

et al. 2021

Procedings Do XV Simpósio Brasileiro De Automação Inteligente

View full text Add to dashboard Cite

In industry, real-time fault detection and diagnosis methods are required to secure processes, reduce damage to products, and avoid possible system failures. Recently, Long Short-Term Memory (LSTM) neural networks are used as an approach to fault detection in industrial process operation because of their strength sequential data processing, such as time series processing. However, LSTM neural networks demand more effort computational to inferring and training when compared to other kinds of neural network architectures. Then, considering IIoT (Industrial Internet of Things) embedded systems have limited memory capacity and small battery charges, strategies to speed up inference in LSTM neural networks and enhance their performance became necessary. In this way, this paper proposes a basis to compress LSTM neural networks with pruning techniques in software. Our pruning approach removes redundant parameters of the LSTM neural network by zeroing absolute synaptic weight values below a threshold. Then, we retrain the pruned model to readjust nonzero weights. We used the Tennessee Eastman Process benchmark to assess our approach. Finally, the paper presents the accuracy, precision, recall and F1-Score for both faulty data sets, varying the network's sparsity and comparing sparsities with performance parameters of the proposed network.

show abstract