Cognito: Automated Feature Engineering for Supervised Learning

Khurana, Udayan; Turaga, Deepak S.; Samulowitz, Horst; Srinivasan, Parthasrathy

doi:10.1109/icdmw.2016.0190

Cited by 124 publications

(88 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Thus, it avoids the drawback of generating a large set of features at each step in [5], especially for complex features and large number of features. One component of CAFEM called FE Learner (FeL) uses Reinforcement Learning to find the optimal feature set F * for each feature iteratively, instead of using expensive graph search algorithm [6]. FeL focus on one particular supervised learning task which gives FeL the ability to dig deeply into that task.…”

Section: Methodsmentioning

confidence: 99%

Cross-data Automatic Feature Engineering via Meta-learning and Reinforcement Learning

Zhang

Hao

Fogelman-Soulié³

2020

Advances in Knowledge Discovery and Data Mining

View full text Add to dashboard Cite

Feature Engineering (FE) is one of the most beneficial, yet most difficult and time-consuming tasks of machine learning projects, and requires strong expert knowledge. It is thus significant to design generalized ways to perform FE. The primary difficulties arise from the multiform information to consider, the potentially infinite number of possible features and the high computational cost of feature generation and evaluation. We present a framework called Cross-data Automatic Feature Engineering Machine (CAFEM), which formalizes the FE problem as an optimization problem over a Feature Transformation Graph (FTG). CAFEM contains two components: a FE learner (FeL) that learns finegrained FE strategies on one single dataset by Double Deep Q-learning (DDQN) and a Cross-data Component (CdC) that speeds up FE learning on an unseen dataset by the generalized FE policies learned by Meta-Learning on a collection of datasets. We compare the performance of FeL with several existing state-of-the-art automatic FE techniques on a large collection of datasets. It shows that FeL outperforms existing approaches and is robust on the selection of learning algorithms. Further experiments also show that CdC can not only speed up FE learning but also increase learning performance.

show abstract

Section: Methodsmentioning

confidence: 99%

Cross-data Automatic Feature Engineering via Meta-learning and Reinforcement Learning

Zhang

Hao

Fogelman-Soulié³

2020

Advances in Knowledge Discovery and Data Mining

View full text Add to dashboard Cite

show abstract

“…Researchers usually encounter a lot challenges in applying deep learning in their research, for example " which deep learning framework shall I use", " how to prepare the data", " how to choose a good model architecture" and "how to set the hyper parameters". AutoML aim to solve these problems by combining many advanced technologies like automated data clean 78 , automated feature engineering 79 , hyper parameter optimization 80 and neural architecture search [40][41][42][43] . Some AutoML tools have been developed like AutoKeras, Auto-sklearn, H2O AutoML, but currently they are only able to search MLP, CNN or RNN based architectures.…”

Section: Autogenome -An Automl Tool For Genomic Researchmentioning

confidence: 99%

AutoGenome: An AutoML Tool for Genomic Research

Liu

et al. 2019

Preprint

View full text Add to dashboard Cite

Deep learning have made great successes in traditional fields like computer vision (CV), natural language processing (NLP) and speech processing. Those achievements greatly inspire researchers in genomic study and make deep learning in genomics a very hot topic. Convolutional neural network (CNN) and recurrent neural network (RNN) are frequently used for genomic sequence prediction problems; multiple layer perception (MLP) and auto-encoders (AE) are frequently used for genomic profiling data like RNA expression data and gene mutation data.Here, we introduce a new neural network architecture, named residual fully-connected neural network (RFCN) and demonstrate its advantage for modeling genomic profiling data. We further incorporate AutoML algorithms and implement AutoGenome, an end-to-end automated genomic deep learning framework. By utilizing the proposed RFCN architectures, automatic hyperparameter search and neural architecture search algorithms, AutoGenome can train highperformance deep learning models for various kinds of genomic profiling data automatically. To make researchers better understand the trained models, AutoGenome can assess the feature importance and export the most important features for supervised learning tasks, and the representative latent vectors for unsupervised learning tasks. We envision AutoGenome to become a popular tool in genomic studies.In the last decades, the emergence of high throughput sequencing technology revolutionized the biomedical research and generated tons of omics data. In genomics area, microarray 1 and next generation DNA-Seq 2 are widely used to identify genome-wide copy number variations, singlenucleotide polymorphism (SNP) and DNA mutations; in epigenomics area, MeDip-Seq 3 , BS-Seq 4 are used to profile DNA methylations; ChIP-Seq is used to identify chromatin associate proteins 5 ; in transcriptomics area, microarray 1 and RNA-Seq 6 are used to quantify whole RNA expressions profile; in proteomics area; LC-MS 7 and ICAT 8 are used to analyze protein complex and quantify proteins; in metabolomics area, MNR 9 and mass spectrometry 10 are used to profile metabolic markers. Omics data provide comprehensive information at different molecular system levels, and have been widely used in biomedical researches 11,12 , and at the same time numerous bioinformatics tools been developed for analyzing omics data.Deep learning have been proved to be very effective in areas like computer vision 13,14 , natural language processing 15 and speech processing [16][17][18] . By leveraging properly designed, deep, stacked neural network architecture, low/middle/high level features could be extracted automatically and combined together to predict the learning target in an end-to-end fashion. Inspired from the achievements of deep learning in the traditional areas, researchers are now actively designing neural network architectures for genomic study, which makes deep learning in genomics study a very hot topic (Fig 1a).Convolutional neural network 14 (CNN) and recurrent neural network 1...

show abstract

“…It searches new numerical features using information gain in a decision tree, given an input dataset and a list of allowed transformations. Likewise, Cognito [7] is based on a transformation tree aiming at recommending a series of transformations. The tree is explored depending on the current performances obtained within the branches.…”

Section: Related Work a Feature Constructionmentioning

confidence: 99%

Consistent Feature Construction with Constrained Genetic Programming for Experimental Physics

Cherrier

Poli

Defurne

et al. 2019

2019 IEEE Congress on Evolutionary Computation (CEC)

View full text Add to dashboard Cite

A good feature representation is a determinant factor to achieve high performance for many machine learning algorithms in terms of classification. This is especially true for techniques that do not build complex internal representations of data (e.g. decision trees, in contrast to deep neural networks). To transform the feature space, feature construction techniques build new high-level features from the original ones. Among these techniques, Genetic Programming is a good candidate to provide interpretable features required for data analysis in high energy physics. Classically, original features or higher-level features based on physics first principles are used as inputs for training. However, physicists would benefit from an automatic and interpretable feature construction for the classification of particle collision events.Our main contribution consists in combining different aspects of Genetic Programming and applying them to feature construction for experimental physics. In particular, to be applicable to physics, dimensional consistency is enforced using grammars.Results of experiments on three physics datasets show that the constructed features can bring a significant gain to the classification accuracy. To the best of our knowledge, it is the first time a method is proposed for interpretable feature construction with units of measurement, and that experts in high-energy physics validate the overall approach as well as the interpretability of the built features.

show abstract

Cognito: Automated Feature Engineering for Supervised Learning

Cited by 124 publications

References 5 publications

Cross-data Automatic Feature Engineering via Meta-learning and Reinforcement Learning

Cross-data Automatic Feature Engineering via Meta-learning and Reinforcement Learning

AutoGenome: An AutoML Tool for Genomic Research

Consistent Feature Construction with Constrained Genetic Programming for Experimental Physics

Contact Info

Product

Resources

About