DBSVEC: Density-Based Clustering Using Support Vector Expansion

Wang, Zhen; Zhang, Rui; Qi, Jianzhong; Yuan, Bo

doi:10.1109/icde.2019.00033

Cited by 13 publications

(6 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Many pretext tasks were found to be conducive to learn image features. For example, image colorization [40,41], super-resolution [21], image processing [39,3], jigsaw puzzles [26], rotation angle prediction [12] and unsupervised deep clustering [2,34]. These methods can learn desired and transferable representations that achieve promising results in downstream tasks.…”

Section: Self-supervised Learning With Pretext Tasksmentioning

confidence: 99%

Contrastive Learning with Diverse Samples

Wu,

Liu

2023

Frontiers in Artificial Intelligence and Applications

View full text Add to dashboard Cite

Unsupervised visual representation learning has gained much attention from the computer vision community because of the recent contrastive learning achievements. Current work mainly adopts instance discrimination as the pretext task, which treats every single instance as a different class (negative) and uses a collection of data augmentation techniques to generate more examples (positive) for each class. The idea is straightforward and efficient but will generally cause similar instances to be classified into different classes. Such problem has been defined as “class collision” in some previous works and is shown to hurt the representation ability. Motivated by this observation, we present a solution to address this issue by filtering similar negative examples from each mini-batch. Concretely, we model the problem as a Determinantal Point Process (DPP) so that similar instances can be filtered stochastically, and diverse samples are expected to be sampled for contrastive training. Besides, we further introduce a priority term for each instance, which indicates the hardness of its positives, so that instances with more hard positives are more likely to be sampled for contributing to the optimization. Our sampling can be efficiently implemented in a feed-forward manner and further accelerated by our encouraged complement DPP. Extensive experimental results demonstrate our priority over the standard setup of contrastive learning.

show abstract

Section: Self-supervised Learning With Pretext Tasksmentioning

confidence: 99%

Contrastive Learning with Diverse Samples

Wu,

Liu

2023

Frontiers in Artificial Intelligence and Applications

View full text Add to dashboard Cite

show abstract

“…Instead, we use embeddings as the network outputs, which naturally allow for modeling emerging new classes, and do not require direct changes to the network structure. Embedding networks map data into a low-dimensional output, where similar data are clustered together and dissimilar data are far apart (Chopra et al 2005;Wang et al 2019bWang et al , 2021a. In the learned space, general metrics, such as L2-distance, can be applied to determine the similarities between the original data.…”

Section: Deep Retrieval For Embeddingsmentioning

confidence: 99%

Continual Learning through Retrieval and Imagination

Wang

Liu

Duan

et al. 2022

AAAI

Self Cite

View full text Add to dashboard Cite

Continual learning is an intellectual ability of artificial agents to learn new streaming labels from sequential data. The main impediment to continual learning is catastrophic forgetting, a severe performance degradation on previously learned tasks. Although simply replaying all previous data or continuously adding the model parameters could alleviate the issue, it is impractical in real-world applications due to the limited available resources. Inspired by the mechanism of the human brain to deepen its past impression, we propose a novel framework, Deep Retrieval and Imagination (DRI), which consists of two components: 1) an embedding network that constructs a unified embedding space without adding model parameters on the arrival of new tasks; and 2) a generative model to produce additional (imaginary) data based on the limited memory. By retrieving the past experiences and corresponding imaginary data, DRI distills knowledge and rebalances the embedding space to further mitigate forgetting. Theoretical analysis demonstrates that DRI can reduce the loss approximation error and improve the robustness through retrieval and imagination, bringing better generalizability to the network. Extensive experiments show that DRI performs significantly better than the existing state-of-the-art continual learning methods and effectively alleviates catastrophic forgetting.

show abstract

“…Emotion: Support vector machine (Atkinson and Campos 2016;Wang et al 2019) and random forest (Liu et al 2016) are first explored for two-category classification. Besides, (Tuncer, Dogan, and Subasi 2021) proposed a fractal pattern feature extraction approach for emotion recognition.…”

Section: Motor Imaginary (Mi)mentioning

confidence: 99%

Cross Task Neural Architecture Search for EEG Signal Classifications

Duan¹,

Wang²,

Li³

et al. 2022

Preprint

View full text Add to dashboard Cite

Electroencephalograms (EEGs) are brain dynamics measured outside of the brain, which have been widely utilized in non-invasive brain-computer interface applications. Recently, various neural network approaches have been proposed to improve the accuracy of EEG signal recognition. However, these approaches severely rely on manually designed network structures for different tasks which normally are not sharing the same empirical design cross-task-wise. In this paper, we propose a cross-task neural architecture search (CTNAS-EEG) framework for EEG signal recognition, which can automatically design the network structure across tasks and improve the recognition accuracy of EEG signals. Specifically, a compatible search space for cross-task searching and an efficient constrained searching method is proposed to overcome challenges brought by EEG signals. By unifying structure search on different EEG tasks, this work is the first to explore and analyze the searched structure difference in crosstask-wise. Moreover, by introducing architecture search, this work is the first to analyze model performance by customizing model structure for each human subject. Detailed experimental results suggest that the proposed CTNAS-EEG could reach state-of-the-art performance on different EEG tasks, such as Motor Imagery (MI) and Emotion recognition. Extensive experiments and detailed analysis are provided as a good reference for follow-up researchers. We will release the source code 1 to contribute to the open source community.

show abstract

DBSVEC: Density-Based Clustering Using Support Vector Expansion

Cited by 13 publications

References 27 publications

Contrastive Learning with Diverse Samples

Contrastive Learning with Diverse Samples

Continual Learning through Retrieval and Imagination

Cross Task Neural Architecture Search for EEG Signal Classifications

Contact Info

Product

Resources

About