Towards Benchmarking Feature Type Inference for AutoML Platforms

Shah, Vraj; Lacanlale, Jonathan; Kumar, Premanand; Yang, Kevin; Kumar, Arun

doi:10.1145/3448016.3457274

Cited by 15 publications

(3 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our core explanation methods rely on fitting an appropriate explanation methodology to data types we find in the origin O and the goal G. Rather than the traditional database attribute types (strings, integers, floats, etc. ), given the nature of our analysis, we look into ML feature types [89]. We focus on three main types, namely, Numeric, Categorical and Textual (mixed types are considered textual), which characterizes the core changes Explain-Da-V covers.…”

Section: Core Semantic Explanation Methodsmentioning

confidence: 99%

Explaining Dataset Changes for Semantic Data Versioning with Explain-Da-V (Technical Report)

Shraga¹,

Miller²

2023

Preprint

View full text Add to dashboard Cite

In multi-user environments in which data science and analysis is collaborative, multiple versions of the same datasets are generated. While managing and storing data versions has received some attention in the research literature, the semantic nature of such changes has remained under-explored. In this work, we introduce Explain-Da-V, a framework aiming to explain changes between two given dataset versions. Explain-Da-V generates explanations that use data transformations to explain changes. We further introduce a set of measures that evaluate the validity, generalizability, and explainability of these explanations. We empirically show, using an adapted existing benchmark and a newly created benchmark, that Explain-Da-V generates better explanations than existing data transformation synthesis methods.

show abstract

Section: Core Semantic Explanation Methodsmentioning

confidence: 99%

Explaining Dataset Changes for Semantic Data Versioning with Explain-Da-V (Technical Report)

Shraga¹,

Miller²

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Several benchmarks are published in the NAS area called NAS-Bench 101 [81], 201 [22] and 301 [68]. There are also benchmarks for some stages in the data science life cycle such as data cleaning [49] and feature type recognition [65]. Different from existing work, we are the first to benchmark automated feature processing.…”

Section: Research Opportunitiesmentioning

confidence: 99%

“…Fortunately, many research efforts have been devoted to answering this question. They conduct comprehensive surveys or experiments on feature selection [20,26,47], hyperparameter tuning [24,80], data cleaning [49], feature type inference [65], etc. However, feature preprocessing, an essential task for classical ML, has not been well explored in the literature.…”

Section: Introductionmentioning

confidence: 99%

FRIEND: Feature selection on inconsistent data

Wang

et al. 2020

Neurocomputing

View full text Add to dashboard Cite

AutoML in heavily constrained applications

Neutatz,

Lindauer,

Abedjan

2023

The VLDB Journal

View full text Add to dashboard Cite

Optimizing a machine learning pipeline for a task at hand requires careful configuration of various hyperparameters, typically supported by an AutoML system that optimizes the hyperparameters for the given training dataset. Yet, depending on the AutoML system’s own second-order meta-configuration, the performance of the AutoML process can vary significantly. Current AutoML systems cannot automatically adapt their own configuration to a specific use case. Further, they cannot compile user-defined application constraints on the effectiveness and efficiency of the pipeline and its generation. In this paper, we propose Caml, which uses meta-learning to automatically adapt its own AutoML parameters, such as the search strategy, the validation strategy, and the search space, for a task at hand. The dynamic AutoML strategy of Caml takes user-defined constraints into account and obtains constraint-satisfying pipelines with high predictive performance.

show abstract

Towards Benchmarking Feature Type Inference for AutoML Platforms

Cited by 15 publications

References 12 publications

Explaining Dataset Changes for Semantic Data Versioning with Explain-Da-V (Technical Report)

Explaining Dataset Changes for Semantic Data Versioning with Explain-Da-V (Technical Report)

FRIEND: Feature selection on inconsistent data

AutoML in heavily constrained applications

Contact Info

Product

Resources

About