Discriminative neural network pruning in a multiclass environment: A case study in spoken emotion recognition

Sánchez-Gutiérrez, Máximo Eduardo; González-Pérez, Pedro Pablo

doi:10.1016/j.specom.2020.03.006

Cited by 12 publications

(9 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The advantages of pruning a bigger network were discussed in a recent article [ 5 ]. The question arises as to whether the solution to an audio signal multi-class classification task is still helpful to medical data and medical data analytics.…”

Section: Materials and Methodsmentioning

confidence: 99%

“…The question arises as to whether the solution to an audio signal multi-class classification task is still helpful to medical data and medical data analytics. The algorithm, as described in [ 5 ], requires a RBM trained in an unsupervised fashion to obtain the propagated values for each training sample from the different classes in the dataset. The outputs of each hidden neuron are then used to calculate their discriminative value.…”

Section: Materials and Methodsmentioning

confidence: 99%

“…Our proposal aims to investigate the post-training neuronal pruning methodology using dissimilarity measures inspired in the information-entropy theory. In a previous paper [ 5 ], we presented the conceptual framework for discriminative pruning based on statistical distances to calculate each RBM’s hidden neuron’s utility in a classification problem. Here, we broaden the approach to classifying multiple classes within the study of medical data characterising various cancer forms.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Multi-Class Classification of Medical Data Based on Neural Network Pruning and Information-Entropy Measures

Sánchez-Gutiérrez

González-Pérez

2022

Entropy

View full text Add to dashboard Cite

Medical data includes clinical trials and clinical data such as patient-generated health data, laboratory results, medical imaging, and different signals coming from continuous health monitoring. Some commonly used data analysis techniques are text mining, big data analytics, and data mining. These techniques can be used for classification, clustering, and machine learning tasks. Machine learning could be described as an automatic learning process derived from concepts and knowledge without deliberate system coding. However, finding a suitable machine learning architecture for a specific task is still an open problem. In this work, we propose a machine learning model for the multi-class classification of medical data. This model is comprised of two components—a restricted Boltzmann machine and a classifier system. It uses a discriminant pruning method to select the most salient neurons in the hidden layer of the neural network, which implicitly leads to a selection of features for the input patterns that feed the classifier system. This study aims to investigate whether information-entropy measures may provide evidence for guiding discriminative pruning in a neural network for medical data processing, particularly cancer research, by using three cancer databases: Breast Cancer, Cervical Cancer, and Primary Tumour. Our proposal aimed to investigate the post-training neuronal pruning methodology using dissimilarity measures inspired by the information-entropy theory; the results obtained after pruning the neural network were favourable. Specifically, for the Breast Cancer dataset, the reported results indicate a 10.68% error rate, while our error rates range from 10% to 15%; for the Cervical Cancer dataset, the reported best error rate is 31%, while our proposal error rates are in the range of 4% to 6%; lastly, for the Primary Tumour dataset, the reported error rate is 20.35%, and our best error rate is 31%.

show abstract

Section: Materials and Methodsmentioning

confidence: 99%

Section: Materials and Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Multi-Class Classification of Medical Data Based on Neural Network Pruning and Information-Entropy Measures

Sánchez-Gutiérrez

González-Pérez

2022

Entropy

View full text Add to dashboard Cite

show abstract

“…Both the growing and pruning approach based architectures repeat three key operations until acceptable performance (low complexity) is achieved [108]: i) training the model, ii) changing the weights (parameters) based on the growing or pruning criteria, and iii) re-training the model. In recent years, the field of growing and pruning has received considerable attention from the research community and several studies have discussed its effectiveness in various research domains, including health service improvement [109], self-care activities [108], and speech emotion recognition [110]. Therefore, the implementation of growing and pruning approaches for multiscale models in a smart city environment is still an open direction for researchers and industry.…”

Section: Growing and Pruning Multiscale Modelsmentioning

confidence: 99%

Multiscale Modeling in Smart Cities: A Survey on applications, Current Trends, and Challenges

Khan¹,

Aurangzeb²,

Aslam³

et al. 2021

Preprint

View full text Add to dashboard Cite

Megacities are complex systems facing the challenges of overpopulation, poor urban design and planning, poor mobility and public transport, poor governance, climate change issues, poor sewerage and water infrastructure, waste and health issues, and unemployment. Smart cities have emerged to address these challenges by making the best use of space and resources for the benefit of citizens. A smart city model views the city as a complex adaptive system consisting of services, resources, and citizens that learn through interaction and change in both the spatial and temporal domains. The characteristics of dynamic development and complexity are key issues for city planners that require a new systematic and modeling approach. Multiscale modeling (MM) is an approach that can be used to better understand complex adaptive systems. The MM aims to solve complex problems at different scales, i.e., micro, meso, and macro, to improve system efficiency and mitigate computational complexity and cost. In this paper, we present an overview of MM in smart cities. First, this study discusses megacities, their current challenges, and their emergence to smart cities. Then, we discuss the need of MM in smart cities and its emerging applications. Finally, the study highlights current challenges and future directions related to MM in smart cities, which provide a roadmap for the optimized operation of smart city systems.

show abstract

“…Pruning is the process of removing unused connections and, eventually, neurons from a network. It has been widely used in a range of speech processing tasks, for example in speech recognition [10,11,12]; denoising, and enhancement [13,14,15]; and, emotion recognition [16]. The efficacy of quantisation, a lowering in the resolution of a network's weights and biases, has been established in similar applications [17,18,19].…”

Section: Introductionmentioning

confidence: 99%

Squeeze for Sneeze: Compact Neural Networks for Cold and Flu Recognition

Albes¹,

Ren²,

Schuller³

et al. 2020

Interspeech 2020

View full text Add to dashboard Cite

In digital health applications, speech offers advantages over other physiological signals, in that it can be easily collected, transmitted, and stored using mobile and Internet of Things (IoT) technologies. However, to take full advantage of this positioning, speech-based machine learning models need to be deployed on devices that can have considerable memory and power constraints. These constraints are particularly apparent when attempting to deploy deep learning models, as they require substantial amounts of memory and data movement operations. Herein, we test the suitability of pruning and quantisation as two methods to compress the overall size of neural networks trained for a health-driven speech classification task. Key results presented on the Upper Respiratory Tract Infection Corpus indicate that pruning, then quantising a network can reduce the number of operational weights by almost 90 %. They also demonstrate the overall size of the network can be reduced by almost 95 %, as measured in MB, without affecting overall recognition performance.

show abstract

Discriminative neural network pruning in a multiclass environment: A case study in spoken emotion recognition

Cited by 12 publications

References 19 publications

Multi-Class Classification of Medical Data Based on Neural Network Pruning and Information-Entropy Measures

Multi-Class Classification of Medical Data Based on Neural Network Pruning and Information-Entropy Measures

Multiscale Modeling in Smart Cities: A Survey on applications, Current Trends, and Challenges

Squeeze for Sneeze: Compact Neural Networks for Cold and Flu Recognition

Contact Info

Product

Resources

About