WSEM _QT : a novel approach for quality‐based evaluation of web data sources for a data warehouse

Abstract

The incorporation of suitable external data from the World Wide Web offers an effective solution for enriching the data in the data warehouse (DW). However, the main challenge is the quality‐aware selection of web data sources to maintain the quality of the DW. In the previous works, the quality evaluation of web sources is through expert evaluation only, which makes it a very lengthy process. Also, since the quality model consists of mixed quality factors from diverse domains of Web, DW and underlying business, finding an expert possessing an expertise of all these domains is a huge bottleneck in the evaluation process. In order to overcome these existing issues, this study proposes a novel multi‐level approach web source evaluation with multi‐criteria decision‐making and web quality testing tools (WSEMQT) and underlying quality model web quality model for evaluating web sources for the DW. The authors introduce automated web source quality evaluation in the first level of web source based evaluation and multiple dimensions of quality evaluation at the second level of expert‐based evaluation. At both the levels, multi‐criteria decision‐making methods are applied to the evaluation scores obtained to ascertain the ranked list of Web sources. The authors present a real‐world academic web data case study which shows that the proposed approach can be executed successfully for real‐world problems.

Empirical Validation of WebQMDW Model for Quality-based External Web Data Source Incorporation in a Data Warehouse

“…Due to the peculiar characteristics of Web portals as opposed to a traditional software product, a plethora of research works have specifically addressed the Website/Web portal quality [31]- [33]. The quality evaluation of WDSs for a DW, however, encompasses the data quality as well as the source Website quality because due attention needs to be given to the quality requirements specific to the destination of the WDS incorporation, i.e., the Data Warehouse and the underlying business domain as well [27]. Few of the important works in the area of defining quality factors/models for using the WDSs as EDS for the DW are summarized in Table I.…”

“…It is a known fact that the quality of the DW data sources hugely impacts the quality of the DW itself [1], [2]. This fact makes the quality-aware evaluation and selection of high quality, credible, and compatible Web Data Sources (WDSs) a very crucial task in the incorporation of Web data in the DW [4], [17]- [27]. There are, however, many challenges in this task like the availability of a massive amount of information, the heterogeneous structure and format of the Web Data [4], the dynamic nature [17], and poor reliability [18] of a significant chunk of Web Sources.…”

“…To the best of the authors' knowledge, none of these quality evaluation models for evaluating WDSs as external data sources for a DW have been empirically validated to corroborate their applicability in this problem area. In order to fill this research gap, in this paper, we present the empirical validation of the state-of-the-art multi-level, multidimensional WebQMDW (Web quality model for evaluating web sources for the DW) quality model [27] to enhance the decision making in a Data Warehouse. WebQMDW model [27] is the first of its kind model which segregates the quality factors in such a way to introduce automated quality evaluation as screening (at the first level) and separation of expert evaluation of different expert areas into different dimensions (at the second level).…”

“…In order to fill this research gap, in this paper, we present the empirical validation of the state-of-the-art multi-level, multidimensional WebQMDW (Web quality model for evaluating web sources for the DW) quality model [27] to enhance the decision making in a Data Warehouse. WebQMDW model [27] is the first of its kind model which segregates the quality factors in such a way to introduce automated quality evaluation as screening (at the first level) and separation of expert evaluation of different expert areas into different dimensions (at the second level). The present work complements and extends the authors' previous work [27] of the quality-based evaluation of the websites of academic institutions for incorporation as WDSs in a University DW.…”

“…WebQMDW model [27] is the first of its kind model which segregates the quality factors in such a way to introduce automated quality evaluation as screening (at the first level) and separation of expert evaluation of different expert areas into different dimensions (at the second level). The present work complements and extends the authors' previous work [27] of the quality-based evaluation of the websites of academic institutions for incorporation as WDSs in a University DW. The said work proposed and used the novel WSEM QT (Web source evaluation with multi-criteria decision-making methods and web quality testing tools) process in conjunction with the underlying novel WebQMDW quality model.…”