Performance evaluation of DCT and wavelet transform for LSI

Jaber, Tareq; Amira, Abbes; Milligan, Peter

doi:10.1109/iscas.2008.4542041

Cited by 3 publications

(5 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…P recision is the fraction of the documents retrieved that are relevant to the user's query, and recall The SVD image is slightly brighter, and has more dark pixels which represent zeros. For HWT image, the great deal of sparseness (dark pixels) has been reduced and values are redistributed more evenly is the fraction of the documents that are relevant to the query and successfully retrieved [5].…”

Section: Results and Analysismentioning

confidence: 99%

“…Further the idea of the TDM as a gray scale image has also been postulated, and the equivalence of using the HWT to remove lexical noise and using the HWT to remove noise from an image has been discussed [4]. The aim of the research presented in this paper, and continuing on the research work in [5], is to develop a new approach to the LSI process based on the possibility of using image processing techniques in text document retrieval. In particular, the effect of using the HWT as a pre-processing step to the SVD is studied.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Latent Semantic Indexing Using Multiresolution Analysis

Jaber¹,

Amira²,

Milligan³

2011

Proceedings of the 1st International Conference on Pervasive and Embedded Computing and Communication Systems

View full text Add to dashboard Cite

Latent semantic indexing (LSI) is commonly used to match queries to documents in information retrieval (IR) applications. It has been shown to improve the retrieval performance, as it can deal with synonymy and polysemy problems. This paper proposes a hybrid approach which can improve result accuracy significantly. Evaluation of the approach based on using the Haar wavelet transform (HWT) as a preprocessing step for the singular value decomposition (SVD) in the LSI system is presented, using Donoho s thresholding with the transformation in HWT. Furthermore, the effect of different levels of decomposition in the HWT process is investigated. The experimental results presented in the paper confirm a significant improvement in performance by applying the HWT as a preprocessing step using Donoho s thresholding.

show abstract

Section: Results and Analysismentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Latent Semantic Indexing Using Multiresolution Analysis

Jaber¹,

Amira²,

Milligan³

2011

Proceedings of the 1st International Conference on Pervasive and Embedded Computing and Communication Systems

View full text Add to dashboard Cite

show abstract

“…The most common properties that are widely accepted by the research community are recall and precision [30]. Precision is the fraction of the documents retrieved that are relevant to the user's query, and recall is the fraction of the documents that are relevant to the query and successfully retrieved [31]. Both recall and precision are needed for measuring issues in the IR.…”

Section: Metrics Usedmentioning

confidence: 99%

Enhanced approach for latent semantic indexing using wavelet transform

Jaber

Amira

Milligan

2012

IET Image Processing

View full text Add to dashboard Cite

Latent semantic indexing (LSI) is a technique used for intelligent information retrieval (IR). It can be used as an alternative to traditional keyword matching IR and is attractive in this respect because of its ability to overcome problems with synonymy and polysemy. This study investigates various aspects of LSI: the effect of the Haar wavelet transform (HWT) as a preprocessing step for the singular value decomposition (SVD) in the key stage of the LSI process; and the effect of different threshold types in the HWT on the search results. The developed method allows the visualisation and processing of the term document matrix, generated in the LSI process, using HWT. The results have shown that precision can be increased by applying the HWT as a preprocessing step, with better results for hard thresholding than soft thresholding, whereas standard SVD-based LSI remains the most effective way of searching in terms of recall value.

show abstract

“…The query intersects the TDM at the first and last documents, and as mentioned before the TDM is diagonal and therefore the S matrix is also diagonal, and so removing any elements from the diagonal values results in removing the same diagonal values in the original TDM. Thus, when the SVD is applied at the range of k−values (1)(2)(3)(4)(5)(6)(7)(8)(9)(10)(11)(12)(13)(14), only the first document is returned, because at this range of values the last document, which is document number 15, is ignored. It is important to note that, applying the LSI at k − value 15, which means no dimension reduction occurs, returns the two documents, the first and the last documents in the TDM, which have a cosine value of 0.7071.…”

Section: Ii-c the Tdm Diagonalmentioning

confidence: 99%

“…We proposed in [11] a new approach to the LSI process based on the possibility of using image processing techniques in text document retrieval. In this paper, we will continue the research work and focus on different aspects of LSI e.g.…”

Section: Introductionmentioning

confidence: 99%

Textual noise analysis and removal for effective search engines

Jaber

Amira

Milligan

2010

2010 2nd European Workshop on Visual Information Processing (EUVIP)

Self Cite

View full text Add to dashboard Cite

In the field of intelligent information retrieval (IR), latent semantic indexing (LSI) is a popular technique used to retrieve information related more in meaning than in lexical matching. A core component in the process is the use of the singular value decomposition (SVD) which is used to remove the lexical noise in the term document matrix (TDM). The topic of mathematical modelling for noise reduction in LSI is important and demands attention. In this paper some observations on aspects of this topic are introduced. The work addresses a definition for noise in text processing and seeks to determine the best structure for the TDM. In other words, the structure of the TDM that would facilitate efficient searching within the LSI.

show abstract

Performance evaluation of DCT and wavelet transform for LSI

Cited by 3 publications

References 11 publications

Latent Semantic Indexing Using Multiresolution Analysis

Latent Semantic Indexing Using Multiresolution Analysis

Enhanced approach for latent semantic indexing using wavelet transform

Textual noise analysis and removal for effective search engines

Contact Info

Product

Resources

About