Examining and improving the effectiveness of relevance feedback for retrieval of scanned text documents

Abstract

Important legacy paper documents are digitized and collected in online accessible archives. This enables the preservation, sharing, and significantly the searching of these documents. The text contents of these document images can be transcribed automatically using OCR systems and then stored in an information retrieval system. However, OCR systems make errors in character recognition which have previously been shown to impact on document retrieval behaviour. In particular relevance feedback query-expansion methods, which are often effective for improving electronic text retrieval, are observed to be less reliable for retrieval of scanned document images. Our experimental examination of the effects of character recognition errors on an ad hoc OCR retrieval task demonstrates that, while baseline information retrieval can remain relatively unaffected by transcription errors, relevance feedback via query expansion becomes highly unstable. This paper examines the reason for this behaviour, and introduces novel modifications to standard relevance feedback methods. These methods are shown experimentally to improve the effectiveness of relevance feedback for errorful OCR transcriptions. The new methods combine similar recognised character strings based on term collection frequency and a string edit-distance measure. The techniques are domain independent and make no use of external resources such as dictionaries or training data.

Managing misspelled queries in IR applications

“…Practical results suggest that while baseline IR can remain relatively unaffected by misspellings, relevance feedback via query expansion becomes highly unstable under these conditions (Lam-Adesina and Jones, 2006). This constitutes a major drawback in the design of IR systems, since query expansion is a major issue in the production of improved query formulations (Guo and Ramakrishnan, 2009;Lu et al, 2009a,b;Stokes et al, 2009).…”

Using String Comparison in Context for Improved Relevance Feedback in Different Text Media

“…In previous work [1] we demonstrated that the reduction in PRF performance for DIR is due to selection of some expansion terms with very low n(i) values which are misrecognized versions of more common terms corrupted at the character level. We could of course try to correct these errors in post-processimg with a dictionary, however existing work [14] indicates that this can introduce more problems for information retrieval due to false substitutions than it solves.…”

“…In previous work [1], we demonstrated that a simple filtering of terms with low n(i) values partially addresses the problems with PRF for DIR associated with spelling mistakes illustrated in Table 1. However, the optimal value of n(i) for filtering may be sensitive to the statistics of individual collections.…”

“…This result motivated us to investigate both the reasons for the ineffectiveness of PRF for DIR, and more generally to seek to understand the reasons for the variations in PRF effectiveness for different text media. In a previous study [1], we showed that the principal problem for PRF in DIR is the presence in the collection of high numbers of very rare index terms which are in fact character corrupted versions of standard words. While correctly spelled versions of these words have standard expected word frequencies within the collection as a whole.…”

“…Current transcription technologies achieve good performance on tasks such as read speech and recently printed texts, but still have very significant error levels for more challenging tasks such as conversational speech in noisy environments and nth generation photocopies or hand written texts. It can be observed that even the level of typographical errors found in published electronic texts can be sufficient to be detrimental to relevance feedback [1].…”