Analysis of biomedical and health queries: Lessons learned from<scp>TREC</scp>and<scp>CLEF</scp>evaluation benchmarks

Tamine, Lynda; Chouquet, Cécile; Palmer, Thomas

doi:10.1002/asi.23351

Cited by 13 publications

(10 citation statements)

References 54 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In a clinical information search setting, the shorter the query in terms of words with low hierarchical specificity (refers to "is-a" specificity derived from a medical terminology), the more difficult it is [19].…”

Section: Research Contributions and Hypothesesmentioning

confidence: 99%

“…x is the set of Nc weighted concepts associated to query facet Qx resulting from Algorithm 1, SIM (c, d) is the cosine similarity between the TF-IDF vectors of document d and preferred entry of concept c [19,15]. With respect to the prioritized aggregation operator principle [11] and according to research hypothesis H3, we compute the PICO importance weights as follows:…”

Section: Computing the Document Relevance Scoresmentioning

confidence: 99%

“…• (Main) Steps 1-13 : Given a word-based PICO query Q, the related annotation QP ICO , the faceted queries QP , QIC and QO and the list D * N (in document collection C) of N d top ranked documents that answer query Q, the algorithm first builds the semantic subgraphs GP , GIC and GO after (1) extracting from a medical terminology (eg., MeSH), the active concepts of each faceted query, respectively Concepts(QP ), Concepts(QIC ) and Concepts(QO), using a concept extraction method (eg., [19,15]); each active concept c is considered at relative level 0 and has an importance score Score(c) that highlights the likelihood of similarity between the concept preferred entry and the query words; (2) building the associated graphs GP , GIC and GO (based on respectively Concepts(QP ), Concepts(QIC ) and Concepts(QO)) by appending to the active concepts the corresponding hypernyms through terminology function HypG processed on medical terminology T until reaching the first common concept. Figure 3 illustrates the results of this step on query Q given in the introduction where the active concepts of the query and the related scores are highlighted in bold.…”

mentioning

confidence: 97%

See 2 more Smart Citations

Aggregating semantic information nuggets for answering clinical queries

Znaidi

Tamine

Latiri

2016

Proceedings of the 31st Annual ACM Symposium on Applied Computing

Self Cite

View full text Add to dashboard Cite

In this paper, we address the issue of answering PICO 1 clinical queries formulated within the Evidence Based Medicine framework. Answering clinical questions gives raise to numerous challenges among wich term ambiguity and relevane estimation based on the distribution of the query facets in the documents. The contributions of this work include (1) a new algorithm for query refinement based on the semantic mapping of each facet of the query to a reference terminology and (2) a new document ranking model based on a prioritized aggregation operator that leverages the importance of each facet with regard to a candidate relevant document. The effectiveness of our PICO-based search approach is empirically evaluated using a clinical retrieval collection including 423 queries and more than 1.2 million of medical abstracts from PubMed. The experimental results show that our approach for PICO query answering significantly overpasses state-of-the-art document ranking models.

show abstract

Section: Research Contributions and Hypothesesmentioning

confidence: 99%

Section: Computing the Document Relevance Scoresmentioning

confidence: 99%

mentioning

confidence: 97%

See 1 more Smart Citation

Aggregating semantic information nuggets for answering clinical queries

Znaidi

Tamine

Latiri

2016

Proceedings of the 31st Annual ACM Symposium on Applied Computing

Self Cite

View full text Add to dashboard Cite

show abstract

“…IR performance evaluation involves test collections, sampling, topics (queries, tasks) formation, and relevance evaluation, and as a general topic, this area has been widely studied (Corcoglioniti, Dragoni, Rospocher, & Aprosio, 2016;Cormack & Lynam, 2006;Hu, Huang, & Hu, 2012;J€ arvelin & Kek€ al€ ainen, 2002;Koopman, Bruza, Sitbon, & Lawley, 2011;Liu, An, & Huang, 2015;Tamine, Chouquet, & Palmer, 2015;Waitelonis, Exeler, & Sack, 2015;Yilmaz, Kanoulas, & Aslam, 2008). In this article, we study relevance evaluation, and particularly, novelty and diversity evaluation in biomedical IR.…”

Section: Related Workmentioning

confidence: 99%

geNov: A new metric for measuring novelty and relevancy in biomedical information retrieval

Huang

2017

Asso for Info Science & Tech

View full text Add to dashboard Cite

For diversity and novelty evaluation in information retrieval, we expect that the novel documents are always ranked higher than the redundant ones and the relevant ones higher than the irrelevant ones. We also expect that the level of novelty and relevancy should be acknowledged. Accordingly, we expect that the evaluation algorithm would reward rankings that respect these expectations. Nevertheless, there are few research articles in the literature that study how to meet such expectations, even fewer in the field of biomedical information retrieval. In this article, we propose a new metric for novelty and relevancy evaluation in biomedical information retrieval based on an aspect-level performance measure introduced by TREC Genomics Track with formal results to show that those expectations above can be respected under ideal conditions. The empirical evaluation indicates that the proposed metric, geNov, is greatly sensitive to the desired characteristics above, and the three parameters are highly tuneable for different evaluation preferences. By experimentally comparing with state-of-the-art metrics for novelty and diversity, the proposed metric shows its advantages in recognizing the ranking quality in terms of novelty, redundancy, relevancy, and irrelevancy and in its discriminative power. Experiments reveal the proposed metric is faster to compute than state-of-the-art metrics.

show abstract

“…Several information retrieval (IR) studies (Hauff, Azzopardi, & Hiemstra, ; Tamine, Chouquet, & Palmer, ) have adopted features such as term frequency and query length to predict the effectiveness of query and retrieval systems. Ayadi et al () and Bashir and Rauber () used these features to predict a correlation between query and retrieval function; Burges et al (), Can, Croft, and Manmatha (), Cao, Qin, Liu, Tsai, and Li (), and Ye and Huang () used them to learn to rank, and Xu, Xu, Wang, and Wang () used them to re‐rank.…”

Section: Related Work: Query Featuresmentioning

confidence: 99%

Mining correlations between medically dependent features and image retrieval models for query classification

Ayadi

Torjmen-Khemakhem

Daoud

et al. 2017

Asso for Info Science & Tech

View full text Add to dashboard Cite

The abundance of medical resources has encouraged the development of systems that allow for efficient searches of information in large medical image data sets. State‐of‐the‐art image retrieval models are classified into three categories: content‐based (visual) models, textual models, and combined models. Content‐based models use visual features to answer image queries, textual image retrieval models use word matching to answer textual queries, and combined image retrieval models, use both textual and visual features to answer queries. Nevertheless, most of previous works in this field have used the same image retrieval model independently of the query type. In this article, we define a list of generic and specific medical query features and exploit them in an association rule mining technique to discover correlations between query features and image retrieval models. Based on these rules, we propose to use an associative classifier (NaiveClass) to find the best suitable retrieval model given a new textual query. We also propose a second associative classifier (SmartClass) to select the most appropriate default class for the query. Experiments are performed on Medical ImageCLEF queries from 2008 to 2012 to evaluate the impact of the proposed query features on the classification performance. The results show that combining our proposed specific and generic query features is effective in query classification.

show abstract

Analysis of biomedical and health queries: Lessons learned fromTRECandCLEFevaluation benchmarks

Cited by 13 publications

References 54 publications

Aggregating semantic information nuggets for answering clinical queries

Aggregating semantic information nuggets for answering clinical queries

geNov: A new metric for measuring novelty and relevancy in biomedical information retrieval

Mining correlations between medically dependent features and image retrieval models for query classification

Contact Info

Product

Resources

About