A non-learning approach to spelling correction in web queries

Soo, Jason

doi:10.1145/2487788.2487834

Cited by 3 publications

(5 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This suggests that 2 may be the best candidate vector size for most applications. This is supported by previous research ( [24], [19]) which show that a balance of context-free and context-dependent candidates perform best. By using a fusion method with a candidate vector size of 2, we select 2 candidates based on context (selected using bigrams) and 2 candidates not based on context (selected using Segments' substrings rules).…”

Section: A Methodssupporting

confidence: 86%

Searching Corrupted Document Collections

Soo

2016

2016 12th IAPR Workshop on Document Analysis Systems (DAS)

Self Cite

View full text Add to dashboard Cite

Historical documents are typically digitized using optical Character Recognition. While effective, the results may not always be accurate and are highly dependent on the input. Consequently, degraded documents are often corrupted. Our focus is finding flexible, reliable methods to correct for such degradation, in the face of limited resources. We extend upon our substring and context fusion based retrieval system known as Segments, to consider metadata. By extracting topics from documents, and supplementing and weighting our lexicon with co-occurring terms found in documents with those topics, we achieve a statistically significant improvement over the state-ofthe-art in all but one test configuration. Our mean reciprocal rank measured on two free, publicly available, independently judged datasets is 0.7657 and 0.5382.

show abstract

Section: A Methodssupporting

confidence: 86%

Searching Corrupted Document Collections

Soo

2016

2016 12th IAPR Workshop on Document Analysis Systems (DAS)

Self Cite

View full text Add to dashboard Cite

show abstract

“…However, Segments's ability to rank is limited by its lack of context knowledge. When context is important, Segments performs poorly [20]. Once k = 5, the ranking becomes less important (which Segments is poor at), and selection becomes more important (which Segments is good at).…”

Section: Methodsmentioning

confidence: 99%

“…For completeness, Segments is a system that takes an input string, and using 6 substring rules, returns a list of possible correction candidates derived from a lexicon, ranked by similarity. A detailed Segments description is found in [21,22,20,23]. Recent research has reaffirmed the potential of segmenting strings by using said segments to perform authorship attribution [19].…”

Section: Segmentsmentioning

confidence: 99%

Revisiting Known-Item Retrieval in Degraded Document Collections

Soo

2016

Self Cite

View full text Add to dashboard Cite

Optical character recognition software converts an image of text to a text document but typically degrades the document's contents. Correcting such degradation to enable the document set to be queried effectively is the focus of this work. The described approach uses a fusion of substring generation rules and context aware analysis to correct these errors. Evaluation was facilitated by two publicly available datasets from TREC-5's Confusion Track containing estimated error rates of 5% and 20%. On the 5% dataset, we demonstrate a statistically significant improvement over the prior art and Solr's mean reciprocal rank (MRR). On the 20% dataset, we demonstrate a statistically significant improvement over Solr, and have similar performance to the prior art. The described approach achieves an MRR of 0.6627 and 0.4924 on collections with error rates of approximately 5% and 20% respectively.

show abstract

“…This collection supports comparison with their work and evaluates in context‐aware situations. Our selection processes has been outlined further by Soo (). USHMM Names, selected names from the Yizkor book collections and victim records (Amir, ). It is a collection of 13 different languages, allowing evaluation of the proposed approach on different and diverse languages. USHMM Spoken Names Query Log, foreign name user query logs.…”

Section: Discussionmentioning

confidence: 99%

“…In general, supervised algorithms outperform unsupervised algorithms, particularly in cases in which context is important in correcting a word (Lim, ); however, they cannot be used in the absence of training data. We describe an unsupervised approach that has no dependence on domain, language structure, or sequential windows (Soo, ; Soo & Frieder, ). The proposed solution outperforms prior unsupervised solutions and is comparable with a leading supervised approach.…”

Section: Introductionmentioning

confidence: 99%