Interactive Elicitation of Knowledge on Feature Relevance Improves Predictions in Small Data Sets

Micallef, Luana; Sundin, Iiris; Marttinen, Pekka; Ammad-ud-din, Muhammad; Peltola, Tomi; Soare, Marta; Kaski, Samuel

doi:10.48550/arxiv.1612.02487

Cited by 1 publication

(2 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Moreover, the difference between the MSE values obtained by the interactive interface and the non-interactive interface shows the amount of improvement made to the predictions using the interface. To test the statistical significance of the improvements made by our method compared to each of the other methods, we used the same procedure introduced in [13]. The distance between the average curves in the last round (round 20 in Figure 3) is used as the test statistics.…”

Section: User Studymentioning

confidence: 99%

See 1 more Smart Citation

Interactive Prior Elicitation of Feature Similarities for Small Sample Size Prediction

Afrabandpey

Peltola

Kaski

2017

Proceedings of the 25th Conference on User Modeling, Adaptation and Personalization

Self Cite

View full text Add to dashboard Cite

Regression under the "small n, large p" conditions, of small sample size n and large number of features p in the learning data set, is a recurring setting in which learning from data is difficult. With prior knowledge about relationships of the features, p can effectively be reduced, but explicating such prior knowledge is difficult for experts. In this paper we introduce a new method for eliciting expert prior knowledge about the similarity of the roles of features in the prediction task. The key idea is to use an interactive multidimensionalscaling (MDS) type scatterplot display of the features to elicit the similarity relationships, and then use the elicited relationships in the prior distribution of prediction parameters. Specifically, for learning to predict a target variable with Bayesian linear regression, the feature relationships are used to construct a Gaussian prior with a full covariance matrix for the regression coefficients. Evaluation of our method in experiments with simulated and real users on text data confirm that prior elicitation of feature similarities improves prediction accuracy. Furthermore, elicitation with an interactive scatterplot display outperforms straightforward elicitation where the users choose feature pairs from a feature list.

show abstract

Section: User Studymentioning

confidence: 99%

“…The user's knowledge assumed to be about feature relevance and/or feature weight values. Similarly, Micallef et al [13] proposed an interactive visualization to extract user's knowledge on the relevance of individual features for a prediction task.…”

mentioning

confidence: 99%