Effective Bayesian-network-based missing value imputation enhanced by crowdsourcing

Ye, Chen; Wang, Hongzhi; Lu, Wenbo; Li, Jianzhong

doi:10.1016/j.knosys.2019.105199

Cited by 17 publications

(6 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To exemplify, for human collaboration in the first category of the data pre-processing approaches, some studies are described as follows. To cope with missing values in data set, Ye et al [ 54 ] proposed a Bayesian-network based missing value imputation approach that was enhanced by crowdsourcing. Crowdsourcing was introduced as an external source where ordinary people can provide contextual knowledge and use their cognitive ability to deal with missing values.…”

Section: Resultsmentioning

confidence: 99%

A Review on Human–AI Interaction in Machine Learning and Insights for Medical Applications

Maadi

Khorshidi

Aickelin

2021

IJERPH

View full text Add to dashboard Cite

Objective: To provide a human–Artificial Intelligence (AI) interaction review for Machine Learning (ML) applications to inform how to best combine both human domain expertise and computational power of ML methods. The review focuses on the medical field, as the medical ML application literature highlights a special necessity of medical experts collaborating with ML approaches. Methods: A scoping literature review is performed on Scopus and Google Scholar using the terms “human in the loop”, “human in the loop machine learning”, and “interactive machine learning”. Peer-reviewed papers published from 2015 to 2020 are included in our review. Results: We design four questions to investigate and describe human–AI interaction in ML applications. These questions are “Why should humans be in the loop?”, “Where does human–AI interaction occur in the ML processes?”, “Who are the humans in the loop?”, and “How do humans interact with ML in Human-In-the-Loop ML (HILML)?”. To answer the first question, we describe three main reasons regarding the importance of human involvement in ML applications. To address the second question, human–AI interaction is investigated in three main algorithmic stages: 1. data producing and pre-processing; 2. ML modelling; and 3. ML evaluation and refinement. The importance of the expertise level of the humans in human–AI interaction is described to answer the third question. The number of human interactions in HILML is grouped into three categories to address the fourth question. We conclude the paper by offering a discussion on open opportunities for future research in HILML.

show abstract

Section: Resultsmentioning

confidence: 99%

A Review on Human–AI Interaction in Machine Learning and Insights for Medical Applications

Maadi

Khorshidi

Aickelin

2021

IJERPH

View full text Add to dashboard Cite

show abstract

“…Since histograms are often maintained for query optimization and approximate processing [26,41] such a technique is non-blocking. Strategies that use master data [19,38,42,43] are also non-blocking since they look up a knowledge base and crowd source the imputations one tuple (or a set of tuples) at a time. Imputation strategies in time series data [14,28,32] are often performed by learning patterns over historical data to forecast current missing values or using the correlation across the time series.…”

Section: Preliminaries 21 Imputation Operationmentioning

confidence: 99%

QUIP: Query-driven Missing Value Imputation

Lin¹,

Mehrotra²

2022

Preprint

View full text Add to dashboard Cite

This paper develops a query-time missing value imputation framework, entitled QUIP, that minimizes the joint costs of imputation and query execution. QUIP achieves this by modifying how relational operators are processed. It adds a cost-based decision function in each operator that checks whether the operator should invoke imputation prior to execution or to defer the imputations for downstream operators to resolve. QUIP implements a new approach to evaluating outer join that preserve missing values during query processing, and a bloom filter based index structure to optimize the space and running overhead. We have implemented QUIP using ImputeDB -a specialized database engine for data cleaning. Extensive experiments on both real and synthetic data sets demonstrates the effectiveness and efficiency of QUIP, which outperforms the state-of-the-art ImputeDB by 2 to 10 times on different query sets and data sets, and achieves the order-of-magnitudes improvement over offline approach.

show abstract

“…In terms of investment, complete data makes it easy for investors to make decisions quickly. Research related to missing values has been done a lot including for clustering [8], multivariate time series [25], active learning [9], a novel weighted distance threshold method [5], electronic health records [23], air pollution [18], financial statement fraud [6], software performance of components [4], mining gradual patterns [19], mail survey [16], spatial [3], obstetrics clinical data [2], autoencoder [12], transfer learning [14], encoder signals [26], Bayesian network [24], industry [22], etc.…”

Section: Introductionmentioning

confidence: 99%

The Best K-Exponential Moving Average with Missing Values: Gold Prices in Indonesia, Saudi Arabia, and Turkey during COVID-19

Mubarak

Aslanargun

Şiklar

2022

icdsos

View full text Add to dashboard Cite

There have been missing values in the gold price data for Indonesia, Saudi Arabia, and Turkey at the weekend so that imputation techniques have been carried out to solve this problem. The imputation method of replacing NAs with the latest non-NA values also known as last observation carried forward (LOCF) made it a solution to overcome the missing values. This study selected the best -exponential moving average based on the smallest mean absolute percentage error (MAPE) from simulations. The 2-exponential moving average analysis was the best analysis for the price of gold which has missing values in Indonesia, Saudi Arabia, and Turkey during COVID-19, while the largest MAPE values are different for each country.

show abstract

Effective Bayesian-network-based missing value imputation enhanced by crowdsourcing

Cited by 17 publications

References 31 publications

A Review on Human–AI Interaction in Machine Learning and Insights for Medical Applications

A Review on Human–AI Interaction in Machine Learning and Insights for Medical Applications

QUIP: Query-driven Missing Value Imputation

The Best K-Exponential Moving Average with Missing Values: Gold Prices in Indonesia, Saudi Arabia, and Turkey during COVID-19

Contact Info

Product

Resources

About