Bayesian compression for dynamically expandable networks

Yang, Yang; Chen, Bin; Liu, Hongwei

doi:10.1016/j.patcog.2021.108260

Cited by 11 publications

(3 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Evaluating and comparing neurosymbolic generative models to regular generative models is crucial for understanding their strengths and weaknesses. This can be achieved through various benchmarks and tasks, as well as by analyzing their performance in different domains and applications [24]. Metrics such as accuracy, interpretability, and efficiency can be used to compare the performance of neurosymbolic generative models to purely symbolic or neural approaches, providing insights into their effectiveness and potential for future research and applications.…”

Section: Evaluation and Comparison Of Neurosymbolic Generative Modelsmentioning

confidence: 99%

Exploring the Synergy of Symbolic and Neural Approaches: Advancements in Neurosymbolic Generative Models for Complex Data Structures and Applications

Srivastava

2023

Preprint

View full text Add to dashboard Cite

In this paper, we explore the synergy of symbolic and neural approaches in the context of neurosymbolic generative models. These models integrate the strengths of symbolic models in representing abstract concepts and logical reasoning with the learning and generalization capabilities of neural models, opening new avenues for dealing with complex data structures and applications. We delve into various aspects of neurosymbolic generative models, including neurosymbolic programming, generative models for discrete structures, probabilistic circuits and tractable models, causality-inspired generative models, and their applications in natural language processing, computer vision, programming, and robotics. Furthermore, we discuss the evaluation and comparison of neurosymbolic generative models to regular generative models, highlighting their potential benefits and challenges. By investigating the advancements in neurosymbolic generative models, we aim to shed light on the future of this interdisciplinary research area and its potential impact on various fields.

show abstract

Section: Evaluation and Comparison Of Neurosymbolic Generative Modelsmentioning

confidence: 99%

Exploring the Synergy of Symbolic and Neural Approaches: Advancements in Neurosymbolic Generative Models for Complex Data Structures and Applications

Srivastava

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…In online learning, when a DNN model is utilized for continuous learning, it uses the lifelong learning concept 11 . Researchers have developed various methods to implement lifelong learning 12 .…”

Section: Related Workmentioning

confidence: 99%

“…In this approach, the model can only increase the neurons in the topmost layer. Furthermore, Yoon et al 11 . have offered another approach of dynamically expandable networks in which neurons are increased at the desired layer.…”

Section: Related Workmentioning

confidence: 99%

Handling catastrophic forgetting using cross-domain order in incremental deep learning

Kumar

Agarwal

2023

J. Electron. Imag.

View full text Add to dashboard Cite

In the present era of big data applications, incremental learning has emerged as the most admired area of research where ever-ending tasks from different application domains arrive temporarily, and the learning models majorly focus on newly arriving tasks while forgetting the historically acquired knowledge of old tasks. In this context, when a deep neural network (DNN) performs a sequence of learning tasks, it usually forgets the previously learned tasks. This phenomenon is known as catastrophic forgetting (CF), and the existing state-of-the-art deep learning approaches are unable to handle this issue. Following this, many DNN-based state-of-the-art solutions have been proposed to alleviate the problem that ensures a more natural way of human brain imitation. During training, deep learning algorithms are subjected to a random sequence of diversified datasets (also referred to as cross-domain datasets) that typically belong to multiple domains and have a variety of feature representation, distribution, density, etc. In such cases, CF may become a serious issue because the model tends to forget the previously learned features that affect the overall performance. Because there has been very little research in this area, a more robust incremental deep learning algorithm is proposed in our study to learn desired tasks incrementally. To overcome this situation, proper sequencing of cross-domain datasets is important so that related datasets are kept together to enhance the learnability of the model. Our research proposes a cross-domain order approach with hard attention to the task (HAT) methodology that uses an attention-gated mechanism to efficiently learn the task-specific features. Intense experiments have been performed, and the performance of the proposed approach is compared with the changing cross-domain orders with the baseline models over the following datasets: MNIST, P-MNIST, CIFAR-10, CIFAR-100, and SVHN, etc. The results of the experiments show that learning through cross-domain orders results in better incremental learning.

show abstract

Multi-modal fusion approaches for tourism: A comprehensive survey of data-sets, fusion techniques, recent architectures, and future directions

Khan,

Ahmad,

Rizwan

et al. 2024

Computers and Electrical Engineering

View full text Add to dashboard Cite

Bayesian compression for dynamically expandable networks

Cited by 11 publications

References 9 publications

Exploring the Synergy of Symbolic and Neural Approaches: Advancements in Neurosymbolic Generative Models for Complex Data Structures and Applications

Exploring the Synergy of Symbolic and Neural Approaches: Advancements in Neurosymbolic Generative Models for Complex Data Structures and Applications

Handling catastrophic forgetting using cross-domain order in incremental deep learning

Multi-modal fusion approaches for tourism: A comprehensive survey of data-sets, fusion techniques, recent architectures, and future directions

Contact Info

Product

Resources

About