Progressive DARTS: Bridging the Optimization Gap for NAS in the Wild

Chen, Xin; Xie, Lingxi; Wu, Jun

doi:10.1007/s11263-020-01396-x

Cited by 41 publications

(26 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Progressive Differentiable Architecture Search(PDARTS) 21 Handcrafted neural networks have traditionally carried out many perceptual tasks by making use of deep learning. The recent emergence of NAS(Neural Architectural Search) has allowed a paradigm shift from manual to automated model design and selection.…”

Section: Methodsmentioning

confidence: 99%

Neural Architecture Search For Skin Cancer Detection

Kandukuri

Sakhtivel

Xie

2021

Preprint

View full text Add to dashboard Cite

Neural Architectural Search (NAS) is a novel method capable of achieving state-of-the-art performance with limited computational resources and time. These coupled factors have resulted in its increasing popularity in many domains. NAS helps to discover an effective architecture for a given task. In parallel, learning through tests, a technique used in human learning aims at improving learning results: a chain of new assessments are conducted with increasing difficulty; the learner uses them to discover susceptible points, and those susceptible points are further addressed to pass the evaluation effectively. When applied in the case of learning in machines, this technique enhances their learning ability and is called Learning by passing tests(LPT). We propose to use the LPT technique in combination with NAS, particularly for Differentiable Architecture Search(DARTS), Progressive Differentiable Architecture Search(PDARTS) and, Partially Connected Differentiable Architecture Search(PCDARTS) to solve the medical challenge of Skin Cancer Classification. A bilevel optimization algorithm is formulated using LPT and is applied on the HAM10000 dataset and the Kaggle Skin Cancer: Malignant Vs. Benign dataset. Our LPT algorithm coupled with NAS can attain better performance than the traditional NAS methods and different state-of-the-art models for the given classification task.

show abstract

Section: Methodsmentioning

confidence: 99%

Neural Architecture Search For Skin Cancer Detection

Kandukuri

Sakhtivel

Xie

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…It introduces the first-& second-order approximation-based approaches according to the calculation of architecture gradient, where the second-order one leads to better performance but lower search speed. However, the DARTS suffers from (i) the performance gap between the search & evaluation phases [4,5], (ii) repeating blocks restriction [3], (iii) performance collapse [9,17,41] due to the model over-fitting, (iv) degenerate architectures [41], and (v) aggregation of skip connections [4,5,8]. Consequently, several works are presented to address the problems of DARTS.…”

Section: Differentiable Nasmentioning

confidence: 99%

“…However, this work exploits the second-order DARTS integrated into a visual tracking framework by adopting a cell-level search procedure. Besides, the operationlevel Dropout [4,5] and a proposed early-stopping strategy are used to alleviate the aggregation of skip connections and select the best cell architecture.…”

Section: Differentiable Nasmentioning

confidence: 99%

“…The candidate operations consist of 3×3 & 5×5 separable convolutions, 3×3 & 5×5 dilated convolutions, 3×3 max pooling, 3×3 average pooling, zero (no connection), and skip connection (i.e., M = 8). Since the DARTS tends to aggregate skip connections due to the rapid error decay during its optimization [4][5], the CHASE employs the operation-level Dropout (previously used by [4,5]) with an initial rate τ, which gradually decays during the search procedure. However, the CHASE does not control the number of skip connections in the final cell architecture (in contrast to [4]).…”

Section: Proposed Approach: Chasementioning

confidence: 99%

“…Moreover, the proposed approach exploits the complete search space of DARTS. Last but not least, the proposed approach does not stack multiple cells to provide a balanced accuracy & speed for visual tracking and address the conventional problem of the performance gap between the search and evaluation phases [5]. Finally, the effectiveness of NAS exploitation and its generalization is validated by employing three versions of DARTS [8,24] and integrating the proposed approach into two visual trackers [1,12].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

CHASE: Robust Visual Tracking via Cell-Level Differentiable Neural Architecture Search

Marvasti-Zadeh¹,

Khaghani²,

Li³

et al. 2021

Preprint

View full text Add to dashboard Cite

A strong visual object tracker nowadays relies on its well-crafted modules, which typically consist of manually-designed network architectures to deliver high-quality tracking results. Not surprisingly, the manual design process becomes a particularly challenging barrier, as it demands sufficient prior experience, enormous effort, intuition and perhaps some good luck. Meanwhile, neural architecture search has gaining grounds in practical applications such as image segmentation, as a promising method in tackling the issue of automated search of feasible network structures. In this work, we propose a novel celllevel differentiable architecture search mechanism to automate the network design of the tracking module, aiming to adapt backbone features to the objective of a tracking network during offline training. The proposed approach is simple, efficient, and with no need to stack a series of modules to construct a network. Our approach is easy to be incorporated into existing trackers, which is empirically validated using different differentiable architecture search-based methods and tracking objectives. Extensive experimental evaluations demonstrate the superior performance of our approach over five commonly-used benchmarks. Meanwhile, our automated searching process takes 41 (18) hours for the second (first) order DARTS method on the TrackingNet dataset.

show abstract

A Higher Performing DARTS Model for CIFAR-10

Shin¹,

Kang²

2023

Intelligent Human Computer Interaction

View full text Add to dashboard Cite

Progressive DARTS: Bridging the Optimization Gap for NAS in the Wild

Cited by 41 publications

References 40 publications

Neural Architecture Search For Skin Cancer Detection

Neural Architecture Search For Skin Cancer Detection

CHASE: Robust Visual Tracking via Cell-Level Differentiable Neural Architecture Search

A Higher Performing DARTS Model for CIFAR-10

Contact Info

Product

Resources

About