Arabic information retrieval

Pseudo-relevance feedback (PRF) is a very effective query expansion approach, which reformulates queries by selecting expansion terms from top k pseudo-relevant documents. Although standard PRF models have been proven effective to deal with vocabulary mismatch between users’ queries and relevant documents, expansion terms are selected without considering their similarity to the original query terms. In this article, we propose a method to incorporate word embedding (WE) similarity into PRF models for Arabic information retrieval (IR). The main idea is to select expansion terms using their distribution in the set of top pseudo-relevant documents along with their similarity to the original query terms. Experiments are conducted on the standard Arabic TREC 2001/2002 collection using three neural WE models. The obtained results show that our PRF extensions significantly outperform their baseline PRF models. Moreover, they enhanced the baseline IR model by 22% and 68% for the mean average precision (MAP) and the robustness index (RI), respectively.

Section: Experimental Evaluationmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Word-embedding-based pseudo-relevance feedback for Arabic information retrieval

Mahdaouy

Alaoui

Gaussier

2018

“…The major constraint in the case of the Arabic language is the limited number of thesauruses or ontological knowledge bases [18]. As a result, most studies on Arabic IR have focused on evaluation or comparison using wordstemming techniques [19][20][21][22] and rank documents based on the stemmed words shared by documents and queries.…”

Section: Introductionmentioning

confidence: 99%

Word-embedding-based query expansion: Incorporating Deep Averaging Networks in Arabic document retrieval

Farhan

Noah

Mohd

et al. 2021

One of the main issues associated with search engines is the query–document vocabulary mismatch problem, a long-standing problem in Information Retrieval (IR). This problem occurs when a user query does not match the content of stored documents, and it affects most search tasks. Automatic query expansion (AQE) is one of the most common approaches used to address this problem. Various AQE techniques have been proposed; these mainly involve finding synonyms or related words for the query terms. Word embedding (WE) is one of the methods that are currently receiving significant attention. Most of the existing AQE techniques focus on expanding the individual query terms rather the entire query during the expansion process, and this can lead to query drift if poor expansion terms are selected. In this article, we introduce Deep Averaging Networks (DANs), an architecture that feeds the average of the WE vectors produced by the Word2Vec toolkit for the terms in a query through several linear neural network layers. This average vector is assumed to represent the meaning of the query as a whole and can be used to find expansion terms that are relevant to the complete query. We explore the potential of DANs for AQE in Arabic document retrieval. We experiment with using DANs for AQE in the classic probabilistic BM25 model as well as for two recent expansion strategies: Embedding-Based Query Expansion approach (EQE1) and Prospect-Guided Query Expansion Strategy (V2Q). Although DANs did not improve all outcomes when used in the BM25 model, it outperformed all baselines when incorporated into the EQE1 and V2Q expansion strategies.

“…There is a need to develop a more refined Arabic IR system handling MSA and dialectical terms in the search queries. For any language, the effectiveness of the query used depends upon the system’s capacity to be compatible with the used language by means of understanding the language characteristics [23]. Therefore we will start by going over the differences between Nejdi (the dialect of our choice for this study) and MSA.…”

Section: Our Proposed Information Retrieval Systemmentioning

confidence: 99%

“…Therefore we manually collected sets of tweets, out of which we compiled a large list of Nejdi dialect words, along with their stem, and the corresponding MSA equivalent word, and the list of dialect stop-words. The latter are a set of common words that do not have a significant meaning in the query [23]. The list of 255 dialectal words and their corresponding MSA words was divided into nine categories.…”

Section: Our Proposed Information Retrieval Systemmentioning

confidence: 99%

Modern information retrieval in Arabic – catering to standard and colloquial Arabic users

Azmi

Aljafari

2015

The widespread use of colloquial dialects among the younger generation of Arabs is depriving many of them the fruits of information freedom. Although most Arabs have no problem with reading text in formal Arabic, widely known as Modern Standard Arabic (MSA), the younger generation is more adept at colloquial Arabic, mainly owing to the widespread use of social media. The current search engines cater mostly to MSA. This means that materials written in colloquial are off-limits to those who use MSA, and similarly the MSA contents are off-limits for those who communicate in colloquial only. To achieve the full potential of an information-retrieval system, we need a successful scheme that interprets queries whether they are in MSA, colloquial Arabic or a combination of both. In this paper we design an information-retrieval system that addresses our concern against the backdrop of one of the local dialects in Saudi Arabia. Our system is based on modifying an MSA stemming technique and a set of colloquial ↔ MSA conversion rules that are lexicon based. We tested the system using 44 queries on a corpus of over 1400 documents (MSA, colloquial, mix). The average precision was 84.3%, while the average recall was 96.5%. In the second test we compared the precision of the retrieved documents by our system vs Google and Yahoo! search engines. The respective average precisions were 78.2, 51.9 and 56.2%.