Automatic caption localization for photographs on World Wide Web pages

Marie-4: a high-recall, self-improving Web crawler that finds images using captions

“…4,6 A sample of our training set showed that 95 percent of the photographs had captions, whereas 10 percent of the nonphotographs had captions. Both can be stored in similar image formats, so some content analysis is necessary to confirm photographs.…”

“…Frew and I previously reported a precision of 0.014 for text queries for a standard browser that tried to find pages with images matching particular words. 4 Recall is harder to estimate, but we got 0.97 in a manual inspection of 20 random Web pages (we defined recall as the fraction of the image-describing text on these pages that was found by our page scanner). The missed caption text was in paragraphs insufficiently related to corresponding images.…”

“…[4][5][6] By looking for the right clues, large amounts of Web page text can be excluded as captions for any given image, and the captions in the remaining text can be inferred. Clues can include caption candidate wording, HTML constructs around the candidate, distance from the associated image, image-file name words, and associated image properties.…”

New use for the pen: outline-based image queries

“…In text-based context search (B), the query method also consists of keywords, but the annotation is automatically derived from the context of the image within the document [6,8]. This approach may be brittle, because only in some domains (e.g., science, journalism) there exists something like a grammar for the juxtaposition of text and image which allows for finding meaningful correspondences between textual and pictorial content.…”

Generic image classification using visual knowledge on the web

“…For each of those images, the system calculates a score that represents the intensity of the relation between the image and the query keywords. Note that an evaluation method used here is based on simple HTML tag analysis, which is similar to a method commonly used in Web image search engines [10,15,16]. The score is calculated by checking the following conditions.…”