A General Hybrid Clustering Technique

Amiri, Saeid; Clarke, Bertrand; Clarke, Jennifer; Koepke, Hoyt

doi:10.1080/10618600.2018.1546593

Cited by 8 publications

(22 citation statements)

References 22 publications

(27 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In general, the clusters in different  ( ) 's do not correspond in a one-to-one manner according to their indices. For instances, (2) 's may be (1) 's with permuted subscript k. To further complicate matters, even permutation may not be a suitable relationship.…”

Section: Consider a Set Of Instancesmentioning

confidence: 99%

“…We will present the method for solving W in the next section. 2 Cluster mapping matrix: Let us take  (1) as the reference partition and attempt to map each (2) ∈  (2) to clusters in  (1) . To mathematically describe the mapping, let i, j be a weight that indicates the extent (2) is mapped to (1) .…”

Section: Consider a Set Of Instancesmentioning

confidence: 99%

“…However, the weights i, j 's can describe more general mappings. For instance, if (2) is split into multiple clusters in  (1) , , > 0 at multiple j's. Define the cluster mapping matrix Γ = ( , ) =1, … , 2 , =1, … , 1 .…”

Section: Consider a Set Of Instancesmentioning

confidence: 99%

“…In the next section, we will describe how to solve Γ. 3 Aligned cluster-posterior matrix: Denote the aligned cluster-posterior matrix of  (2) with respect to  (1) by P (2 → 1) :…”

Section: Consider a Set Of Instancesmentioning

confidence: 99%

“…what should be the correspondence between (1) and (2) ? As discussed in the previous section, a simple type of correspondence is given by permutation.…”

Section: Compute Mean Partition Based On Bootstrap Samplesmentioning

confidence: 99%

See 4 more Smart Citations

Optimal transport, mean partition, and uncertainty assessment in cluster analysis

Seo

Lin

2019

Statistical Analysis

View full text Add to dashboard Cite

In scientific data analysis, clusters identified computationally often substantiate existing hypotheses or motivate new ones. Yet the combinatorial nature of the clustering result, which is a partition rather than a set of parameters or a function, blurs notions of mean, and variance. This intrinsic difficulty hinders the development of methods to improve clustering by aggregation or to assess the uncertainty of clusters generated. We overcome that barrier by aligning clusters via optimal transport. Equipped with this technique, we propose a new algorithm to enhance clustering by any baseline method using bootstrap samples. Cluster alignment enables us to quantify variation in the clustering result at the levels of both overall partitions and individual clusters. Set relationships between clusters such as one‐to‐one match, split, and merge can be revealed. A covering point set for each cluster, a concept kin to the confidence interval, is proposed. The tools we have developed here will help address the crucial question of whether any cluster is an intrinsic or spurious pattern. Experimental results on both simulated and real data sets are provided. The corresponding R package OTclust is available on CRAN.

show abstract

Section: Consider a Set Of Instancesmentioning

confidence: 99%

Section: Consider a Set Of Instancesmentioning

confidence: 99%

Section: Consider a Set Of Instancesmentioning

confidence: 99%

“…In the next section, we will describe how to solve Γ. 3 Aligned cluster-posterior matrix: Denote the aligned cluster-posterior matrix of  (2) with respect to  (1) by P (2 → 1) :…”

Section: Consider a Set Of Instancesmentioning

confidence: 99%

“…what should be the correspondence between (1) and (2) ? As discussed in the previous section, a simple type of correspondence is given by permutation.…”

Section: Compute Mean Partition Based On Bootstrap Samplesmentioning

confidence: 99%

See 3 more Smart Citations

Optimal transport, mean partition, and uncertainty assessment in cluster analysis

Seo

Lin

2019

Statistical Analysis

View full text Add to dashboard Cite

show abstract

Unsupervised and Semisupervised Learning

Pisztora

2021

Wiley StatsRef: Statistics Reference Online

View full text Add to dashboard Cite

Methodologies for unsupervised and semisupervised learning are reviewed. For unsupervised learning, or clustering, the focus is on mixture‐model‐based approaches under both the classic and mode association frameworks. High‐dimensional data pose a major challenge for clustering. We thus discuss in detail variable selection and the hidden Markov model on variable blocks, which exploits a graph structure to simplify the dependence among variables. We also present topics that emerged relatively recently such as clustering distributional data under the Wasserstein metric and uncertainty assessment for cluster analysis. Semisupervised learning has attracted growing interest in the machine learning community in recent years. We review foundational approaches including self‐training, semisupervised generative models, and graphical models. We then describe in greater depth entropy minimization, consistency regularization, and mixup augmentation, methods that are utilized in state‐of‐the‐art models such as MixMatch.

show abstract

Pattern Extraction of Topsoil and Subsoil Heterogeneity and Soil‐Crop Interaction Using Unsupervised Bayesian Machine Learning: An Application to Satellite‐Derived NDVI Time Series and Electromagnetic Induction Measurements

et al. 2019

View full text Add to dashboard Cite

The link between remotely sensed surface vegetation performances with the heterogeneity of subsurface physical properties is investigated by means of a Bayesian unsupervised learning approach. This question has considerable relevance and practical implications for precision agriculture as visible spatial differences in crop development and yield are often directly related to horizontal and vertical variations in soil texture caused by, for example, complex deposition/erosion processes. In addition, active and relict geomorphological settings, such as floodplains and buried paleochannels, can cast significant complexity into surface hydrology and crop modeling. This also requires a better approach to detect, quantify, and analyze topsoil and subsoil heterogeneity and soil‐crop interaction. In this work, we introduce a novel unsupervised Bayesian pattern recognition framework to address the extraction of these complex patterns. The proposed approach is first validated using two synthetic data sets and then applied to real‐world data sets of three test fields, which consists of satellite‐derived normalized difference vegetation index (NDVI) time series and proximal soil measurement data acquired by a multireceiver electromagnetic induction geophysical system. We show, for the first time, how the similarity and joint spatial patterns between crop NDVI time series and soil electromagnetic induction information can be extracted in a statistically rigorous means, and the associated heterogeneity and correlation can be analyzed in a quantitative manner. Some preliminary results from this study improve our understanding the link of above surface crop performance with the heterogeneous subsurface. Additional investigations have been planned for further testing the validity and generalization of these findings.

show abstract

A General Hybrid Clustering Technique

Cited by 8 publications

References 22 publications

Optimal transport, mean partition, and uncertainty assessment in cluster analysis

Optimal transport, mean partition, and uncertainty assessment in cluster analysis

Unsupervised and Semisupervised Learning

Pattern Extraction of Topsoil and Subsoil Heterogeneity and Soil‐Crop Interaction Using Unsupervised Bayesian Machine Learning: An Application to Satellite‐Derived NDVI Time Series and Electromagnetic Induction Measurements

Contact Info

Product

Resources

About