ARIEX: Automated ranking of information extractors

On extracting data from tables that are encoded using HTML

“…Neither is it common to find figures regarding efficiency, which makes it difficult to realise if a proposal might work well in a production scenario. Jiménez et al [29] set a foundation regarding how to evaluate information extraction proposals in general, but they did not focus on the tasks involved in extracting information from tables that are encoded using HTML.…”

On validating web information extraction proposals

“…Furthermore, the validation process is typically poorly documented and there exists much heterogeneity in the experimental settings; in a few cases, no experimental results are reported at all (Baumgartner et al, 2007;Raposo et al, 2002;Sahuguet & Azavant, 2001). Jiménez et al (2016) devised ARIEX, which is the state-of-the-art method to validate information extractors. It allows to check them on a collection of well-known datasets and allows to compare the effectiveness results as homogeneously as possible and to rank them as automatically as possible.…”

“…Summing up, the validations performed in the literature are very diverse and many details have commonly not been unveiled, which makes it difficult to determine which proposal actually performs better than the others. ARIEX was developed on the hope to help researchers validate their proposals (Jiménez et al, 2016), but an in-depth analysis of the literature has revealed three deficiencies that motivated us to work on this article, namely: how complete the annotations in the validation datasets are, how the structure of the extracted information is taken into account, and how the matchings amongst the annotations and the extractions are computed.…”

“…The information extracted by a supervised proposal has user-defined labels with a meaning; contrarily, the information extracted by unsupervised or heuristic-based proposals have computer-generated labels that must be mapped onto user-defined labels later. Jiménez et al (2016) found that the methods used to validate information extractors are often poorly documented and have some common deficiencies that may have a significant impact on the experimental results. This is particularly important insofar it hampers confirming or refuting the results, makes the comparisons with other proposals heterogeneous, and may easily bias the conclusions.…”

Green Information Extraction from Family Books

“…Early researchers within the database, library/information science, document analysis, and AI communities also applied their diverse toolkits to search semi-structured documents like dictionaries [9] and library catalogs [10]. Surveys of the hundreds of ensuing contributions include [11][12][13] and [14], which also evaluate and compare several dozen information extraction systems.…”