Simpler Certified Radius Maximization by Propagating Covariances

Zhen, Xingjian; Chakraborty, Rudrasis; Singh, Vikas

doi:10.1109/cvpr46437.2021.00721

Cited by 3 publications

(1 citation statement)

References 15 publications

(2 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The design of CNNs is affected by a large number of hyperparameters [6,7], which need to be fine-tuned for optimal performance [8]. Previously, studies have concentrated on refining architectures such as VGGNet [9] and ResNet [10], which are either manually crafted by experts or automatically generated through greedy induction methods [6,11]. Despite the remarkable performance of CNN architectures [12,13], specialists in machine learning and optimization propose that improved structures can be produced through automated methods.…”

Section: Introductionmentioning

confidence: 99%

Embedding channel pruning within the CNN architecture design using a bi-level evolutionary approach

Louati

Bechikh

et al. 2023

J Supercomput

View full text Add to dashboard Cite

Remarkable advancements have been achieved in machine learning and computer vision through the utilization of deep neural networks. Among the most advantageous of these networks is the convolutional neural network (CNN). It has been used in pattern recognition, medical diagnosis, and signal processing, among other things. Actually, for these networks, the challenge of choosing hyperparameters is of utmost importance. The reason behind this is that as the number of layers rises, the search space grows exponentially. In addition, all known classical and evolutionary pruning algorithms require a trained or built architecture as input. During the design phase, none of them consider the process of pruning. In order to assess the effectiveness and efficiency of any architecture created, pruning of channels must be carried out before transmitting the dataset and computing classification errors. For instance, following pruning, an architecture of medium quality in terms of classification may transform into an architecture that is both highly light and accurate, and vice versa. There exist countless potential scenarios that could occur, which prompted us to develop a bi-level optimization approach for the entire process. The upper level involves generating the architecture while the lower level optimizes channel pruning. Evolutionary algorithms (EAs) have proven effective in bi-level optimization, leading us to adopt the co-evolutionary migration-based algorithm as a search engine for our bi-level architectural optimization problem in this research. Our proposed method, CNN-D-P (bi-level CNN design and pruning), was tested on the widely used image classification benchmark datasets, CIFAR-10, CIFAR-100 and ImageNet. Our suggested technique is validated by means of a set of comparison tests with regard to relevant state-of-the-art architectures.

show abstract

Section: Introductionmentioning

confidence: 99%

Embedding channel pruning within the CNN architecture design using a bi-level evolutionary approach

Louati

Bechikh

et al. 2023

J Supercomput

View full text Add to dashboard Cite

show abstract

Joint design and compression of convolutional neural networks as a Bi-level optimization problem

Louati

Bechikh

Louati

et al. 2022

Neural Comput & Applic

View full text Add to dashboard Cite

Over the last decade, deep neural networks have shown great success in the fields of machine learning and computer vision. Currently, the CNN (convolutional neural network) is one of the most successful networks, having been applied in a wide variety of application domains, including pattern recognition, medical diagnosis and signal processing. Despite CNNs’ impressive performance, their architectural design remains a significant challenge for researchers and practitioners. The problem of selecting hyperparameters is extremely important for these networks. The reason for this is that the search space grows exponentially in size as the number of layers increases. In fact, all existing classical and evolutionary pruning methods take as input an already pre-trained or designed architecture. None of them take pruning into account during the design process. However, to evaluate the quality and possible compactness of any generated architecture, filter pruning should be applied before the communication with the data set to compute the classification error. For instance, a medium-quality architecture in terms of classification could become a very light and accurate architecture after pruning, and vice versa. Many cases are possible, and the number of possibilities is huge. This motivated us to frame the whole process as a bi-level optimization problem where: (1) architecture generation is done at the upper level (with minimum NB and NNB) while (2) its filter pruning optimization is done at the lower level. Motivated by evolutionary algorithms’ (EAs) success in bi-level optimization, we use the newly suggested co-evolutionary migration-based algorithm (CEMBA) as a search engine in this research to address our bi-level architectural optimization problem. The performance of our suggested technique, called Bi-CNN-D-C (Bi-level convolution neural network design and compression), is evaluated using the widely used benchmark data sets for image classification, called CIFAR-10, CIFAR-100 and ImageNet. Our proposed approach is validated by means of a set of comparative experiments with respect to relevant state-of-the-art architectures.

show abstract