Outlier detection using an ensemble of clustering algorithms

Ray, Bidhan Chandra; Ghosh, Soulib; Ahmed, Shameem; Sarkar, Ram; Nasipuri, Mita

doi:10.1007/s11042-021-11671-9

Cited by 10 publications

(2 citation statements)

References 51 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In 2021, Ray et. al [15] employed an unsupervised learning-based approach by designing an ensemble of three clustering algorithms for outlier detection. Pandey et al [16] proposed a word-embeddings-based approach for anomaly classification.…”

Section: Background and Related Workmentioning

confidence: 99%

Few shot learning with fine-tuned language model for suicidal text detection

Varma¹,

Shivam²,

Ray³

et al. 2022

Preprint

View full text Add to dashboard Cite

The most effective way to prevent potential suicide attempts is early identification followed by prompt treatment. Nowadays, online communication channels especially social media is becoming a way of expressing suicidal tendencies. However, the availability of properly tagged textual data is a major issue for researchers to perform identification or classification of such tendencies through texts. In light of the abovementioned facts, in the present work, we have proposed an approach to determine early detection with the goal of early diagnosis of suicidal behaviour through text posted on social media via supervised learning using few shot learning process. Therefore, to detect suicidal behaviour, we extract embeddings by finetuning a large pre-trained language model. The contextual embeddings have then been used for binary classification of the text for suicidal behaviour, in which a comparison of various classifiers has been performed. This includes traditional supervised classifiers and neural network models. A comparison of the model’s performance with and without an outlier detection and removal step has also been performed to highlight the importance of the outlier detection step in the pipeline. The feasibility and practicality of the approach have been demonstrated by generating results for user-generated content scraped from Reddit platform posts made on subreddits ‘SuicideWatch’ and ‘depression’. The results generated by the model are also seen to outperform various state-of-the-art models.

show abstract

Section: Background and Related Workmentioning

confidence: 99%

Few shot learning with fine-tuned language model for suicidal text detection

Varma¹,

Shivam²,

Ray³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…In addition to pattern matching, the statistical analysis technology and machine learning technology are also used in OD. In recent years, some OD algorithms, such as clustering-based [21,24,27], distance-based [3,13], density-based [28,35,36], model-based [5] have also been used in practical applications of specific domains.…”

Section: Introductionmentioning

confidence: 99%

Detecting Outliers in Data Streams Based on Minimum Rare Pattern Mining and Pattern Matching

Li¹,

Cai

2022

ITC

View full text Add to dashboard Cite

Outliers are the major factors to influence the accuracy of data-based processing, thus, they must be discovered from collected datasets to guarantee data security. With the widely use of sensors and other monitoring equipment, data streams are becoming the main form of data. However, the huge scale of data streams results in the number of mined rare patterns very large, which makes it hard to effectively detect outliers through pattern-based outlier detection methods. Since minimal rare patterns (MRPs) can represent rare patterns and the number is much smaller, therefore, the use of MRPs on outlier detection can short the time consumption. Based on this idea, this paper proposes an outlier detection approach based on minimal rare pattern, called ODMRP, which is composed of pattern mining phase and pattern matching phase. Specifically, in the pattern mining phase, an improved minimal rare pattern mining algorithm, namely MRPM, is proposed to mine the MRPs from data streams; It first constructs two matrix structures to store information of transactions and frequent 2-patterns, and then apply “pattern extension” operations to extend frequent 2-patterns to longer patterns, at the same time, rare patterns are removed to prevent them participating into “pattern extension” operation to reduce meaningless time cost; In the pattern matching phase, we use the IM-Sunday algorithm to match the mined MRPs with the patterns stored in outlier pattern library, to find potential outliers. Extensive experimental studies show that the proposed ODMRP method can accurately detect outliers from data streams in less overhead.

show abstract