Plausibility Testing for Lexical Resources

Abstract

Abstract. This paper describes principles for evaluation metrics for lexical components and an implementation of them based on requirements from practical information systems. Evaluating information system componentsThe performance of a component in a complex processing pipeline can influence the function of downstream components, meaning that end-to-end testing also must be performed on entire systems, using approaches based on use cases with target notions that validate the function of the system for the purpose it is built, such as many of the evaluation measures formulated in workshops at CLEF. But a task-based evaluation does not reveal the performance of individual components. Evaluation of knowledge-based components in an information system should be done systematically, ideally in ways which are similar to unit tests done for other technical components, motivated by the need for a development and maintenance team to:

Social media in the infertile community—using a text analysis tool to identify the topics of discussion on the multitude of infertility blogs

“…After each iteration, the tool recalculated the frequencies of the topics and suggested new topics and associated terms for the revised topics. 36 , 37 Steps 3–4 continued until a consensus was reached and until no further new topics related to the aim were discovered. The software enables analysis of very large amounts of texts which may be difficult to analyze manually.…”

A Rubric to Assess Students' Clinical Reasoning When Encountering Virtual Patients

“…In study II a new tool, the Gavagai Explorer, was used to perform a qualitativequantitative content analysis (Espinoza et al, 2018;Karlgren, 2016;Parks et al, 2017). A strength of Gavagai Explorer is that the tool enables content analysis of large amounts of text and structures the content analysis around what was actually expressed by the respondents and thereby avoiding that the analysis wanders off in directions that is not warranted by the raw data.…”

“…This tool was used to conduct meaning-based text analytics that build on text clustering and topic extraction. It allows for a qualitative descriptive analysis combined with quantification of data (Espinoza et al, 2018;Parks, Karlgren, & Stymne, 2017). Gavagai Explorer performs an analysis of the manifest content by automated interactive lexical text clustering, quantifying and calculating occurrence of topics in the text (Espinoza et al, 2018).…”

“…The tool processes the dataset in order to identify common topics according to term specificity in language. Gavagai Explorer also helps the analyst by suggesting a list of topics based on associated termsand sentiment scores, and ranking the topics based on occurrences in the text (Espinoza et al, 2018;Parks et al, 2017). In order to dig deeper into the text data, terms and topics can be ignored or merged.…”