Decoupled Mixup for Data-efficient Learning

Liu, Zicheng; Liu, Siyuan; Wang, Ge; Tan, Cheng; Wu, Lirong; Li, Stan Z.

doi:10.48550/arxiv.2203.10761

Cited by 2 publications

(2 citation statements)

References 28 publications

(46 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We compared some previous state-of-the-art network architectures with TbsNet ( Table 7 ). These architectures employ some training techniques such as AutoMix ( Liu et al, 2022c ), SAMix ( Li et al, 2021 ), PuzzleMix+DM ( Liu et al, 2022b ), DCL ( Luo et al, 2021 ), etc . Since no training techniques are employed (vanilla training scheme), the TbsNet results on Tiny-ImageNet ( Le & Yang, 2015 ) are not very nice.…”

Section: Benchmarkingmentioning

confidence: 99%

TbsNet: the importance of thin-branch structures in CNNs

Sheng

Shi

et al. 2023

PeerJ Computer Science

View full text Add to dashboard Cite

The performance of a convolutional neural network (CNN) model is influenced by several factors, such as depth, width, network structure, size of the receptive field, and feature map scaling. The optimization of the best combination of these factors poses as the main difficulty in designing a viable architecture. This article presents an analysis of key factors influencing network performance, offers several strategies for constructing an efficient convolutional network, and introduces a novel architecture named TbsNet (thin-branch structure network). In order to minimize computation costs and feature redundancy, lightweight operators such as asymmetric convolution, pointwise convolution, depthwise convolution, and group convolution are implemented to further reduce the network’s weight. Unlike previous studies, the TbsNet architecture design rejects the reparameterization method and adopts a plain, simplified structure which eliminates extraneous branches. We conduct extensive experiments, including network depth, width, etc. TbsNet performs well on benchmark platforms, Top 1 Accuracy on CIFAR-10 is 97.02%, on CIFAR-100 is 83.56%, and on ImageNet-1K is 86.17%. Tbs-UNet’s DSC on the Synapse dataset is 78.39%, higher than TransUNet’s 0.91%. TbsNet can be competent for some downstream tasks in computer vision, such as medical image segmentation, and thus is competitive with prior state-of-the-art deep networks such as ResNet, ResNeXt, RepVgg, ParNet, ConvNeXt, and MobileNet.

show abstract

Section: Benchmarkingmentioning

confidence: 99%

TbsNet: the importance of thin-branch structures in CNNs

Sheng

Shi

et al. 2023

PeerJ Computer Science

View full text Add to dashboard Cite

show abstract

“…Data augmentation is designed based on domain knowledge. For example, in CV datasets, operations such as color jittering [42], random cropping [4], applying Gaussian blur [9], Mixup [24,21,25] are proven useful. In biology and some easy datasets, linear combinations τ lc (•) in k-nearest neighbor data is a simple and effective way.…”

Section: Dlme Frameworkmentioning

confidence: 99%

DLME: Deep Local-flatness Manifold Embedding

Zang¹,

Liu²,

Wu³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Manifold learning (ML) aims to seek low-dimensional embedding from high-dimensional data. The problem is challenging on realworld datasets, especially with under-sampling data, and we find that previous methods perform poorly in this case. Generally, ML methods first transform input data into a low-dimensional embedding space to maintain the data's geometric structure and subsequently perform downstream tasks therein. The poor local connectivity of under-sampling data in the former step and inappropriate optimization objectives in the latter step leads to two problems: structural distortion and underconstrained embedding. This paper proposes a novel ML framework named Deep Local-flatness Manifold Embedding (DLME) to solve these problems. The proposed DLME constructs semantic manifolds by data augmentation and overcomes the structural distortion problem using a smoothness constrained based on a local flatness assumption about the manifold. To overcome the underconstrained embedding problem, we design a loss and theoretically demonstrate that it leads to a more suitable embedding based on the local flatness. Experiments on three types of datasets (toy, biological, and image) for various downstream tasks (classification, clustering, and visualization) show that our proposed DLME outperforms state-of-the-art ML and contrastive learning methods.

show abstract

Decoupled Mixup for Data-efficient Learning

Cited by 2 publications

References 28 publications

TbsNet: the importance of thin-branch structures in CNNs

TbsNet: the importance of thin-branch structures in CNNs

DLME: Deep Local-flatness Manifold Embedding

Contact Info

Product

Resources

About