Text Mining in Official Statistics

Bécue, Mónica; Fridlund, Bert; Fyhrlund, Alf; Prat, Albert; Sundgren, Bo

doi:10.1007/978-3-540-45219-5_15

Cited by 4 publications

(4 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Domain specific data introduces several issues to be addressed. Among these the most important ones are (i) the use of domain specific vocabulary, (ii) existing specific metadata, (iii) various information presentations and (iv) various user backgrounds that scale from unexperienced users to highly experienced ones represented by domain specialists and experts [3].…”

Section: Access To Domain Specific Datamentioning

confidence: 99%

See 1 more Smart Citation

Combining Text Mining and Information Retrieval Techniques for Enhanced Access to Statistical Data on the Web: A Preliminary Report

Rajman

Vesely

2005

Knowledge Mining

View full text Add to dashboard Cite

In this contribution, we present the StatSearch prototype, a search engine that enables an enhanced access to domain specific data available on the Web. The StatSearch engine proposes a hybrid search interface combining query-based search with automated navigation through a tree-like hierarchical structure. The goal of such an interface is to allow a more natural and intuitive control over the information access process, thus improving the speed and quality of the access to information.An algorithm for automated navigation is proposed that requires natural language pre-processing of the documents, including language identification, tokenization, Part-of-Speech (PoS) tagging, lemmatization and entity extraction. Structural transformation of the available data collection is also performed to reorganize the nodes in the information space (the Web site) from a graph into a tree-like hierarchical structure. This structural pre-processing (transformation of a graph structure into a tree-like hierarchy) can be done either by document clustering, or, alternatively, derived from existing structure of the document collection by splitting, shifting, or merging of nodes where necessary. The clustering approach is more straightforward but requires that the intermediate nodes in the created tree are assigned understandable descriptions, which corresponds to a difficult task.Target documents are represented by weighted lexical profiles the components of which correspond to triples of the form (surface form, lemma, PoS). The extracted and normalized terms and entities are weighted using the TF.IDF weighting scheme. Document relevance is computed as the textual similarity between the query and document profiles. Several well known similarity functions from the field of information retrieval have been tested, including the Cosine and Okapi BM25 similarity measures. In addition to the similarity score, the contributions of all the query terms to the computed document similarities are also provided.The principle of the presented algorithm for automated navigation is to compute a score distribution on the documents (leaves of the tree), and to propagate the obtained scores upwards in the tree. The node scores are then used to guide a

show abstract

Section: Access To Domain Specific Datamentioning

confidence: 99%

“…http://europa.eu.int/comm/eurostat/research/ 2 http://www.sdmx.org/3 We focused on the domains of National Accounts, Citizen Influence, Labour Market and Prices and Consumption…”

mentioning

confidence: 99%

Combining Text Mining and Information Retrieval Techniques for Enhanced Access to Statistical Data on the Web: A Preliminary Report

Rajman

Vesely

2005

Knowledge Mining

View full text Add to dashboard Cite

show abstract

“…In that way, a list of potential online platform organizations active in a country could be obtained. Text mining techniques could be used to do this (see Becue et al 2004). In this article, we describe the results of our study which aims to identify the online platform organizations in the Netherlands.…”

Section: Introductionmentioning

confidence: 99%

On the Validity of Using Webpage Texts to Identify the Target Population of a Survey: An Application to Detect Online Platforms

Daas,

Hassink,

Klijs

2024

Journal of Official Statistics

View full text Add to dashboard Cite

A statistical classification model was developed to identify online platform organizations based on the texts on their website. The model was subsequently used to identify all (potential) platform organizations with a website included in the Dutch Business Register. The empirical outcomes of the statistical model were plausible in terms of the words and the bimodal distribution of fitted probabilities, but the results indicated an overestimation of the number of platform organizations. Next, the external validity of the outcomes was investigated through a survey of the organizations that were identified as a platform organization by the statistical classification model. The response by the organizations to the survey confirmed a substantial number of type-I errors. Furthermore, it revealed a positive association between the fitted probability of the text-based classification model and the organization’s response to the survey question on being an online platform organization. The survey results indicated that the text-based classification model can be used to obtain a subpopulation of potential platform organizations from the entire population of businesses with a website. This subpopulation may form a good starting point to study platform organizations in more detail.

show abstract

“…In that way, a list of potential online platform organizations active in a country could be obtained. Text mining techniques could be used to do this (see Becue et al 2004). In this paper, we describe the results of our study which aims to identify the online platform organizations in the Netherlands.…”

Section: Introductionmentioning

confidence: 99%

On the Validity of Using Webpage Texts to Identify the Target Population of a Survey: An Application to Detect Online Platforms

2023

View full text Add to dashboard Cite

Text Mining in Official Statistics

Cited by 4 publications

References 0 publications

Combining Text Mining and Information Retrieval Techniques for Enhanced Access to Statistical Data on the Web: A Preliminary Report

Combining Text Mining and Information Retrieval Techniques for Enhanced Access to Statistical Data on the Web: A Preliminary Report

On the Validity of Using Webpage Texts to Identify the Target Population of a Survey: An Application to Detect Online Platforms

On the Validity of Using Webpage Texts to Identify the Target Population of a Survey: An Application to Detect Online Platforms

Contact Info

Product

Resources

About