Data Science: Similarity, Dissimilarity and Correlation Functions

Batyrshin, Ildar

doi:10.1007/978-3-030-33274-7_2

Cited by 6 publications

(5 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Really, the property of bipolarity of the dissimilarity function ( 20) is surprising because, in the definition of this function, the concept of negation of n-tuples is not used! From the theoretical results of the new theory of correlation functions (association measures) developed in [12][13][14][15][16][17][18][19] and from our previous results, it was shown that Pearson's correlation coefficient could be constructed from (20) by the method given in Theorem 1 due to (20) satisfies co-symmetry and consistency properties. The bipolarity is more strong property than these two properties together, and in Proposition 4, it is shown that bipolarity is also fulfilled for (20).…”

Section: Propositionmentioning

confidence: 99%

“…The concept of correlation function (association measure) was introduced and studied in [12][13][14][15][16][17][18][19]. Let Ω be a nonempty set.…”

Section: Correlation Functions (Association Measures)mentioning

confidence: 99%

“…Example 1. It is shown [12][13][14][15][16][17][18][19] that most of the known in statistics correlation and association coefficients [20][21][22][23][24][25][26] satisfy these properties on specific domain Ω with suitably defined involution operation. Let us show that Pearson's linear correlation coefficient satisfies A1-A3.…”

Section: Correlation Functions (Association Measures)mentioning

confidence: 99%

“…Another reason to consider bipolar similarity functions is that these functions define measures of correlation (association) between elements of the domain. The paper gives a short introduction to the theory of correlation functions (association measures) defined on sets with involutive operation [12][13][14][15][16][17][18][19]. The correlation function (association measure) is defined on such sets as a function satisfying several simple properties.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Bipolar Dissimilarity and Similarity Correlations of Numbers

Batyrshin

Tóth-Laufer

2022

Mathematics

Self Cite

View full text Add to dashboard Cite

Many papers on fuzzy risk analysis calculate the similarity between fuzzy numbers. Usually, they use symmetric and reflexive similarity measures between parameters of fuzzy sets or “centers of gravity” of generalized fuzzy numbers represented by real numbers. This paper studies bipolar similarity functions (fuzzy relations) defined on a domain with involutive (negation) operation. The bipolarity property reflects a structure of the domain with involutive operation, and bipolar similarity functions are more suitable for calculating a similarity between elements of such domain. On the set of real numbers, similarity measures should take into account symmetry between positive and negative numbers given by involutive negation of numbers. Another reason to consider bipolar similarity functions is that these functions define measures of correlation (association) between elements of the domain. The paper gives a short introduction to the theory of correlation functions defined on sets with an involutive operation. It shows that the dissimilarity function generating Pearson’s correlation coefficient is bipolar. Further, it proposes new normalized similarity and dissimilarity functions on the set of real numbers. It shows that non-bipolar similarity functions have drawbacks in comparison with bipolar similarity functions. For this reason, bipolar similarity measures can be recommended for use in fuzzy risk analysis. Finally, the correlation functions between numbers corresponding to bipolar similarity functions are proposed.

show abstract

Section: Propositionmentioning

confidence: 99%

“…The concept of correlation function (association measure) was introduced and studied in [12][13][14][15][16][17][18][19]. Let Ω be a nonempty set.…”

Section: Correlation Functions (Association Measures)mentioning

confidence: 99%

Section: Correlation Functions (Association Measures)mentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Bipolar Dissimilarity and Similarity Correlations of Numbers

Batyrshin

Tóth-Laufer

2022

Mathematics

Self Cite

View full text Add to dashboard Cite

show abstract

“…In this section we use the universal data representation to introduce distances between keyvalue pairs, and emphasise that in contrast to most approaches 7,10,11 we do not rely on the concept of a distance between data records but rather define and employ the distance function between two sets of IDs. In particular, we consider two key-value pairs to be similar, if restricting the data to the corresponding sets of IDs leads to similar answers to subsequent questions.…”

Section: Distances Between Key-value Pairsmentioning

confidence: 99%

Universal distances detect anomalies in big data

Komarov,

Fabian,

Kulikovska

et al. 2024

Preprint

View full text Add to dashboard Cite

The detection of anomalies in a set of data represents a ubiquitous challenge in computer science emerging in a wide range of fields such as financial transactions, particle physics and communication traffic. At the very heart of our method to attack this problem lies a representation of the data by key-value pairs with sets of associated identifiers. We compute distances between sets of key-value pairs with the help of a reference key and associated identifiers, and visualize these distances using heatmaps to bring out with few prior assumptions anomalies. Our method is universal since it works with any tabular or document-oriented data, independent of their content and form. We demonstrate the power of our technique by identifying three anomalies in the New York City Taxi and Limousine Commission Trip Record Data.

show abstract