A comparative evaluation of popular search engines on finding Turkish documents for a specific time period

Abstract

A COMPARATIVE EVALUATION OF POPULAR SEARCH ENGINES ON FINDING TURKISH DOCUMENTS FOR A SPECIFIC TIME PERIOD Yıltan Bitirim, Abdül Kadir GörürOriginal scientific paper This study evaluates the popular search engines, Google, Yahoo, Bing, and Ask, on finding Turkish documents by comparing their current performances with their performances measured six years ago. Furthermore, the study reveals the current information retrieval effectiveness of the search engines. First of all, the Turkish queries were run on the search engines separately. Each retrieved document was classified and precision ratios were calculated at various cut-off points for each query and engine pair. Afterwards, these ratios were compared with the six years ago ratios for the evaluations. Besides the descriptive statistics, Mann-Whitney U and Kruskal-Wallis H statistical tests were used in order to find out statistically significant differences. All search engines, except Google, have better performance today. Bing has the most increased performance compared to six years ago. Nowadays: Yahoo has the highest mean precision ratios at various cut-off points; all search engines have their highest mean precision ratios at cut-off point 5; dead links were encountered in Google, Bing, and Ask; and repeated documents were encountered in Google and Yahoo. Keywords: information retrieval; performance evaluation; search engine; Turkish Usporedna evaluacija popularnih mehanizama za pretraživanje u pronalaženju turskih dokumenata određenog vremenskog razdobljaIzvorni znastveni članak U ovom se istraživanju ocjenjuju popularni mehanizmi za pretraživanje, Google, Yahoo, Bing, i Ask, pri traženju turskih dokumenata usporedbom njihovog sadašnjeg rada s radom izmjerenim prije šest godina. Nadalje, istraživanje pokazuje sadašnju učinkovitost mehanizama u pronalaženju podataka. Najprije su učinjeni upiti za turske riječi odvojeno na svakom mehanizmu. Svaki pronađeni dokument je klasificiran, a izračunati su omjeri točnosti na raznim cut-off točkama za svaki upit i svaki mehanizam. Zatim su ti omjeri uspoređeni s onima od prije šest godina zbog procjene. Pored opisne statistike, korišteni su Mann-Whitney U i Kruskal-Wallis H statistički testovi kako bi se pronašle statistički značajne razlike. Svi mehanizmi za ispitivanje osim Google-a danas su učinkovitiji. Bing je njviše napredovao u odnosu na prije šest godina. Danas Yahoo ima najviše prosječne omjere točnosti u raznim cutoff točkama. Svi mehanizmi za pretraživanje imaju najviše prosječne omjere točnosti u cut-off točki 5; ugašene veze (dead links) su nađene u Google-u, Bingu i Asku, a ponovljeni dokumenti u Google-u i Yahoo-u.

Proposing a New Combined Indicator for Measuring Search Engine Performance and Evaluating Google, Yahoo, DuckDuckGo, and Bing Search Engines based on Combined Indicator

“…In this category of studies, researchers have evaluated the performance of one or several information retrieval system(s) using criteria such as precision. Below are some of these criteria and the research done with them: (a) Expected Search Length Criteria: Chignell et al, 1999; Cooper, 1968; Dunlop, 1997; Silverstein et al, 1998; (b) Precision criteria: Garoufallou, 2012; Griesbaum, 2004; Khan et al, 2014; Ljosland, 1999; Meghabghab and Meghabghab, 1996; Tawileh et al, 2010; (c) Relative recall: Clarke and Willett (1997), (d) Stability criteria: Bar-Ilan, 1998, 2002; (e) Coverage criteria: Henzinger et al (2000, (f) Relative Size & Search Engine Overlap Criteria: Bharat and Broder (1998, g) Click Criterion, Query Log Analysis: Joachims (2002), Joachims et al (2017), Radlinski et al (2008), Dupret and Liao (2010), and Bitirim and Görür (2017).…”