A Lagrangian-based score for assessing the quality of pairwise constraints in semi-supervised clustering

Abstract

Clustering algorithms help identify homogeneous subgroups from data. In some cases, additional information about the relationship among some subsets of the data exists. When using a semi-supervised clustering algorithm, an expert may provide additional information to constrain the solution based on that knowledge and, in doing so, guide the algorithm to a more useful and meaningful solution. Such additional information often takes the form of a cannot-link constraint (i.e., two data points cannot be part of the same cluster) or a must-link constraint (i.e., two data points must be part of the same cluster). A key challenge for users of such constraints in semi-supervised learning algorithms, however, is that the addition of inaccurate or conflicting constraints can decrease accuracy and little is known about how to detect whether expert-imposed constraints are likely incorrect. In the present work, we introduce a method to score each must-link and cannot-link pairwise constraint as likely incorrect. Using synthetic experimental examples and real data, we show that the resulting impact score can successfully identify individual constraints that should be removed or revised.

Exploring dual information in distance metric learning for clustering

“…This constitutes a form of sensitivity analysis (Pichery, 2014) whose goal is to measure the impact of modifications of the input variables on the result of a clustering model. To obtain such information, we have recourse to the Lagrangian duality theory, as explained in Randel et al (2021) The Lagrangian function for the optimization problem ( 3)-( 7) can be obtained by introducing dual variables η c ij , λ c ij and γ c ij to penalize violations of inequality constraints ( 5), ( 6) and ( 6). Specifically, the Lagrangian function L(η, λ, γ) is defined as follows:…”

“…Consequently, the optimal values of the dual variables are reliable estimates of the price to be paid for complying with the constraints associated with them. For more details, the reader is referred to Randel et al (2021).…”

“…where every binary variable y c indicates whether the data point o c is one of the k medoids, and every binary variable x c i indicates whether data point o i is assigned to a cluster having o c as medoid. The Lagrangian function L(η, λ, γ) associated with this optimization problem is then defined as in the previous case by replacing constraints ( 16) and ( 17) by penalty terms in the objective function and the same sub-gradient optimization technique as in (Randel et al, 2021) can be used to determine L D = max η,λ,γ≤0 L(η, λ, γ).…”

“…For each dataset, each set E i of pairwise constraints, and each metric m, we ran the sub-gradient algorithm of Randel et al (2021) to solve (11) and so produce optimal dual variables which give a fitness score as defined in (20). A time limit of 10 seconds has been allocated to each execution of the subgradient algorithm.…”

“…In other words, by associating each pairwise constraint with a dual variable, we can measure the impact of each constraint in the clustering solution. This idea was introduced in our previous work (Randel et al, 2021) with the objective of identifying the presence of erroneous/contradictory pairwise constraints. Motivated by the wealth of information that can be acquired from dual variables, we propose in this paper to further exploit this knowledge by developing tools to help manage the problems described above in distance metric learning for clustering.…”