A Multiple Classifier Approach for the Recognition of Screen-Rendered Text

Wachenfeld, Steffen; Fleischer, Stefan; Jiang, Xiaoyi

doi:10.1007/978-3-540-74272-2_114

Cited by 2 publications

(2 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…If the text is on a background for which Prefab has not been trained, or a complicated background that Prefab cannot model (e.g., a photographic wallpaper), it will not find the text. Recently, computer vision researchers have conducted research on segmentation and recognition methods for small screen-rendered text and reported accuracy achieved of 99.2346% [13,14,15]. However, they assumed the position of the text is known and did not address the problem of text detection.…”

Section: Text Detection and Extraction From Pixelsmentioning

confidence: 95%

“…We did not implement Wachenfeld's screen text recognition algorithms [13,14,15], but used the Tesseract OCR engine (http://code.google.com/p/tesseract-ocr/) in our current prototype instead. If it were given the whole screen image as input, the Tesseract OCR engine would perform poorly because it assumes the text is in a single column.…”

Section: A2 Text Extractionmentioning

confidence: 99%

See 1 more Smart Citation

Associating the visual representation of user interfaces with their internal structures and metadata

Chang

Yeh

Miller

2011

Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology

View full text Add to dashboard Cite

Pixel-based methods are emerging as a new and promising way to develop new interaction techniques on top of existing user interfaces. However, in order to maintain platform independence, other available low-level information about GUI widgets, such as accessibility metadata, was neglected intentionally. In this paper, we present a hybrid framework, PAX, which associates the visual representation of user interfaces (i.e. the pixels) and their internal hierarchical metadata (i.e. the content, role, and value). We identify challenges to building such a framework. We also develop and evaluate two new algorithms for detecting text at arbitrary places on the screen, and for segmenting a text image into individual word blobs. Finally, we validate our framework in implementations of three applications. We enhance an existing pixel-based system, Sikuli Script, and preserve the readability of its script code at the same time. Further, we create two novel applications, Screen Search and Screen Copy, to demonstrate how PAX can be applied to development of desktop-level interactive systems.

show abstract

Section: Text Detection and Extraction From Pixelsmentioning

confidence: 95%

Section: A2 Text Extractionmentioning

confidence: 99%