Scaling up top-K cosine similarity search

Zhu, Shengqi; Wu, Junjie; Xiong, Hui; Xia, Guoping

doi:10.1016/j.datak.2010.08.004

Cited by 41 publications

(21 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…For two individuals whose characteristics have the same dimension number, distance and similarity are usually used to measure the difference between them. The distance measures the absolute distance between two individuals in space which is directly related to the position coordinates of each individual (i.e., the numerical value of each characteristic dimension), but the similarity measures the angle between two individual vectors and reflects more difference in direction that in difference in position [40][41][42]. Therefore, similarity is more suitable to measure the difference between a history year and a forecast year.…”

Section: Similar Years Selectionmentioning

confidence: 99%

“…, n his + n f o ). [40][41][42]. Therefore, similarity is more suitable to measure the difference between a history year and a forecast year.…”

Section: Forecast Model Validity Degreementioning

confidence: 99%

“…Here, the most common cosine similarity is chosen [41,42]. The cosine similarity between the history year HYb and the forecast year FYc is as follows:…”

Section: Forecast Model Validity Degreementioning

confidence: 99%

See 2 more Smart Citations

A Robust Weighted Combination Forecasting Method Based on Forecast Model Filtering and Adaptive Variable Weight Determination

Ding

et al. 2015

Energies

View full text Add to dashboard Cite

Medium-and-long-term load forecasting plays an important role in energy policy implementation and electric department investment decision. Aiming to improve the robustness and accuracy of annual electric load forecasting, a robust weighted combination load forecasting method based on forecast model filtering and adaptive variable weight determination is proposed. Similar years of selection is carried out based on the similarity between the history year and the forecast year. The forecast models are filtered to select the better ones according to their comprehensive validity degrees. To determine the adaptive variable weight of the selected forecast models, the disturbance variable is introduced into Immune Algorithm-Particle Swarm Optimization (IA-PSO) and the adaptive adjustable strategy of particle search speed is established. Based on the forecast model weight determined by improved IA-PSO, the weighted combination forecast of annual electric load is obtained. The given case study illustrates the correctness and feasibility of the proposed method.

show abstract

Section: Similar Years Selectionmentioning

confidence: 99%

“…, n his + n f o ). [40][41][42]. Therefore, similarity is more suitable to measure the difference between a history year and a forecast year.…”

Section: Forecast Model Validity Degreementioning

confidence: 99%

See 1 more Smart Citation

A Robust Weighted Combination Forecasting Method Based on Forecast Model Filtering and Adaptive Variable Weight Determination

Ding

et al. 2015

Energies

View full text Add to dashboard Cite

show abstract

“…Top-k correlated pattern mining was mostly developed only for 2-itemsets [22,23]. Our algorithm produces top-k correlations among itemsets with any number of items.…”

Section: Related Workmentioning

confidence: 99%

Efficient Mining of Top Correlated Patterns Based on Null-Invariant Measures

Kim

Barsky

2011

Machine Learning and Knowledge Discovery in Databases

View full text Add to dashboard Cite

Abstract. Mining strong correlations from transactional databases often leads to more meaningful results than mining association rules. In such mining, null (transaction)-invariance is an important property of the correlation measures. Unfortunately, some useful null-invariant measures such as Kulczynski and Cosine, which can discover correlations even for the very unbalanced cases, lack the (anti)-monotonicity property. Thus, they could only be applied to frequent itemsets as the post-evaluation step. For large datasets and for low supports, this approach is computationally prohibitive. This paper presents new properties for all known null-invariant measures. Based on these properties, we develop efficient pruning techniques and design the Apriori-like algorithm NICOMINER for mining strongly correlated patterns directly. We develop both the threshold-bounded and the top-k variations of the algorithm, where top-k is used when the optimal correlation threshold is not known in advance and to give user control over the output size. We test NICOMINER on real-life datasets from different application domains, using Cosine as an example of the null-invariant correlation measure. We show that NICOMINER outperforms support-based approach more than an order of magnitude, and that it is very useful for discovering top correlations in itemsets with low support.

show abstract

“…However, the traditional supportconfidence framework may not disclose truly interesting relationships [1,2,3]. This is usually illustrated by the well-known "coffee-tea" example [4].…”

Section: Introductionmentioning

confidence: 99%

COSI-: Identification of Cosine Interesting Patterns Based on FP-tree

Huang¹,

Wu²,

Zhu³

et al. 2010

Proceedings of the 2010 International Conference on E-Business Intelligence

Self Cite

View full text Add to dashboard Cite

The cosine similarity, also known as uncentered Pearson Correlation, has been widely used for mining association patterns, which contain objects strongly related to each other. However, it is often used as a post-evaluation measure and is computationally prohibitive for large data. To this end, we develop an FP-tree like algorithm, named COSI-tree, for finding association patterns based on the cosine measure. A key idea is to combine the strength of the FP-tree structure and the Conditional Anti-Monotone Property of the cosine measure. Experimental results on real-world data demonstrate the effectiveness of COSI-tree, in particular for finding rare but interesting patterns at extremely low support levels.

show abstract

Scaling up top-K cosine similarity search

Cited by 41 publications

References 33 publications

A Robust Weighted Combination Forecasting Method Based on Forecast Model Filtering and Adaptive Variable Weight Determination

A Robust Weighted Combination Forecasting Method Based on Forecast Model Filtering and Adaptive Variable Weight Determination

Efficient Mining of Top Correlated Patterns Based on Null-Invariant Measures

COSI-: Identification of Cosine Interesting Patterns Based on FP-tree

Contact Info

Product

Resources

About