Nonstationary data stream classification with online active learning and siamese neural networks✩

Malialis, Kleanthis; Panayiotou, Christos G.; Polycarpou, Marios M.

doi:10.1016/j.neucom.2022.09.065

Cited by 15 publications

(5 citation statements)

References 63 publications

(89 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Among the various approaches that exist, some popular methods are k-nearest neighbors (Aggarwal et al, 2006), decision trees (Domingos & Hulten, 2000), support vector machines (Tsang et al, 2007), fuzzy logic (Das et al, 2016;Iyer et al, 2018), bayesian theory (Seidl et al, 2009) and neural networks (Leite et al, 2013). Recently, deep learning approaches with different learning mechanisms (Hoi et al, 2021) are introduced resulting in architectures like online deep learning (ODL) (Sahoo et al, 2017) and ActiSiamese (Malialis et al, 2022) networks. The ODL and ActiSiamese have shown tremendous improvement in the learning capability for streaming classification tasks.…”

Section: Related Workmentioning

confidence: 99%

Aux-Drop: Handling Haphazard Inputs in Online Learning Using Auxiliary Dropouts

Agarwal¹,

Gupta²,

Horsch³

et al. 2023

Preprint

View full text Add to dashboard Cite

Many real-world applications based on online learning produce streaming data that is haphazard in nature, i.e., contains missing features, features becoming obsolete in time, the appearance of new features at later points in time and a lack of clarity on the total number of input features. These challenges make it hard to build a learnable system for such applications, and almost no work exists in deep learning that addresses this issue. In this paper, we present Aux-Drop, an auxiliary dropout regularization strategy for online learning that handles the haphazard input features in an effective manner. Aux-Drop adapts the conventional dropout regularization scheme for the haphazard input feature space ensuring that the final output is minimally impacted by the chaotic appearance of such features. It helps to prevent the co-adaptation of especially the auxiliary and base features, as well as reduces the strong dependence of the output on any of the auxiliary inputs of the model. This helps in better learning for scenarios where certain features disappear in time or when new features are to be modeled. The efficacy of Aux-Drop has been demonstrated through extensive numerical experiments on SOTA benchmarking datasets that include Italy Power Demand, HIGGS, SUSY and multiple UCI datasets.

show abstract

Section: Related Workmentioning

confidence: 99%

Aux-Drop: Handling Haphazard Inputs in Online Learning Using Auxiliary Dropouts

Agarwal¹,

Gupta²,

Horsch³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…7a). 122 In another recent study, inspired by the concept of siamese neural networks and the DeepCID method, 123,124 a pseudo-siamese convolutional neural network (pSCNN) consisting of two independent CNN subnetworks was proposed for the analysis of the 1 H NMR spectra from mixtures, through which it can determine whether a certain type of compound existing in the mixture. 125…”

Section: In Nmr-based Nps Analysismentioning

confidence: 99%

Machine learning-assisted structure annotation of natural products based on MS and NMR data

Qiu

2023

Nat. Prod. Rep.

View full text Add to dashboard Cite

show abstract

“…Shin et al [ 3 ] proposed a text classification model that detects cyber-security-related tweets by introducing a contrastive word-embedding model that defines positive and negative embedding models to the target event. Malialis et al [ 12 ] proposed a new density-based active learning strategy based on the similarity in the latent space for non-stationary and imbalanced data streams. Nguyen et al [ 7 ] extracted and tracked social events on real-time data streams by aggregating discrete signals representing relevant keywords from the tweets collected by the event categorization.…”

Section: Related Workmentioning

confidence: 99%

“…There have been lots of research efforts to increase the performance of event classification in data streams [ 3 , 5 , 6 , 7 , 8 , 9 , 10 , 11 , 12 ]. Because the performance of the classifier is significantly affected by the underlying embedding models, effective word-embedding methods have been proposed [ 3 , 6 , 11 ].…”

Section: Introductionmentioning

confidence: 99%

“…Because the performance of the classifier is significantly affected by the underlying embedding models, effective word-embedding methods have been proposed [ 3 , 6 , 11 ]. On the other hand, a few studies have investigated building a set of classifiers and selectively utilizing them to respond to non-stationary data streams [ 9 , 10 , 12 ]. However, a scalable architecture for the streaming event classification in a distributed environment, which is required to efficiently respond to detect and track the events from data sources with large volumes and high velocity, has not been considered before.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Effective Model Update for Adaptive Classification of Text Streams in a Distributed Learning Environment

Kim

Lim

Lee

et al. 2022

Sensors

View full text Add to dashboard Cite

In this study, we propose dynamic model update methods for the adaptive classification model of text streams in a distributed learning environment. In particular, we present two model update strategies: (1) the entire model update and (2) the partial model update. The former aims to maximize the model accuracy by periodically rebuilding the model based on the accumulated datasets including recent datasets. Its learning time incrementally increases as the datasets increase, but we alleviate the learning overhead by the distributed learning of the model. The latter fine-tunes the model only with a limited number of recent datasets, noting that the data streams are dependent on a recent event. Therefore, it accelerates the learning speed while maintaining a certain level of accuracy. To verify the proposed update strategies, we extensively apply them to not only fully trainable language models based on CNN, RNN, and Bi-LSTM, but also a pre-trained embedding model based on BERT. Through extensive experiments using two real tweet streaming datasets, we show that the entire model update improves the classification accuracy of the pre-trained offline model; the partial model update also improves it, which shows comparable accuracy with the entire model update, while significantly increasing the learning speed. We also validate the scalability of the proposed distributed learning architecture by showing that the model learning and inference time decrease as the number of worker nodes increases.

show abstract

Nonstationary data stream classification with online active learning and siamese neural networks✩

Cited by 15 publications

References 63 publications

Aux-Drop: Handling Haphazard Inputs in Online Learning Using Auxiliary Dropouts

Aux-Drop: Handling Haphazard Inputs in Online Learning Using Auxiliary Dropouts

Machine learning-assisted structure annotation of natural products based on MS and NMR data

Effective Model Update for Adaptive Classification of Text Streams in a Distributed Learning Environment

Contact Info

Product

Resources

About