Automatic Graph Learning with Evolutionary Algorithms: An Experimental Study

Bu, Chenyang; Lu, Yimin; Liu, Fei

doi:10.1007/978-3-030-89188-6_38

Cited by 6 publications

(4 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The following well-known AutoML tools (listed in alphabetical order) were considered for our study. AutoGL is open source and was created at Tsinghua University for AutoML on graphs and contains four modules, including auto feature engineering, model training, hyperparameter optimization, and auto ensemble [23,24]. AutoGluon is an open-source tool created by Amazon that can automate machine learning and deep learning algorithms for text, images, and datasets [25][26][27].…”

Section: Automl Toolsmentioning

confidence: 99%

Evaluating the Performance of Automated Machine Learning (AutoML) Tools for Heart Disease Diagnosis and Prediction

Paladino,

Hughes,

Perera

et al. 2023

View full text Add to dashboard Cite

Globally, over 17 million people annually die from cardiovascular diseases, with heart disease being the leading cause of mortality in the United States. The ever-increasing volume of data related to heart disease opens up possibilities for employing machine learning (ML) techniques in diagnosing and predicting heart conditions. While applying ML demands a certain level of computer science expertise—often a barrier for healthcare professionals—automated machine learning (AutoML) tools significantly lower this barrier. They enable users to construct the most effective ML models without in-depth technical knowledge. Despite their potential, there has been a lack of research comparing the performance of different AutoML tools on heart disease data. Addressing this gap, our study evaluates three AutoML tools—PyCaret, AutoGluon, and AutoKeras—against three datasets (Cleveland, Hungarian, and a combined dataset). To evaluate the efficacy of AutoML against conventional machine learning methodologies, we crafted ten machine learning models using the standard practices of exploratory data analysis (EDA), data cleansing, feature engineering, and others, utilizing the sklearn library. Our toolkit included an array of models—logistic regression, support vector machines, decision trees, random forest, and various ensemble models. Employing 5-fold cross-validation, these traditionally developed models demonstrated accuracy rates spanning from 55% to 60%. This performance is markedly inferior to that of AutoML tools, indicating the latter’s superior capability in generating predictive models. Among AutoML tools, AutoGluon emerged as the superior tool, consistently achieving accuracy rates between 78% and 86% across the datasets. PyCaret’s performance varied, with accuracy rates from 65% to 83%, indicating a dependency on the nature of the dataset. AutoKeras showed the most fluctuation in performance, with accuracies ranging from 54% to 83%. Our findings suggest that AutoML tools can simplify the generation of robust ML models that potentially surpass those crafted through traditional ML methodologies. However, we must also consider the limitations of AutoML tools and explore strategies to overcome them. The successful deployment of high-performance ML models designed via AutoML could revolutionize the treatment and prevention of heart disease globally, significantly impacting patient care.

show abstract

Section: Automl Toolsmentioning

confidence: 99%

Evaluating the Performance of Automated Machine Learning (AutoML) Tools for Heart Disease Diagnosis and Prediction

Paladino,

Hughes,

Perera

et al. 2023

View full text Add to dashboard Cite

show abstract

“…A majority of works on GML focus on developing new algorithms for certain graph tasks and applications [44,48]. In comparison, there exist relatively few recent works [40,13,47,5,49] that address the GML model selection problem. They mainly focus on neural architecture search and hyperparameter optimization (HPO) for GML models, especially for graph neural networks.…”

Section: Evaluation-based Model Selectionmentioning

confidence: 99%

“…To achieve more efficient model selection than the naive exhaustive approach (Fig. 1b), they investigated techniques for efficient HPO, including subgraph sampling [40], graph coarsening [13], hierarchical evaluation [47], hypernets [49], and evolutionary algorithms [5]. However, for model selection, they still need to perform model training and/or evaluations on the new graph, which is much more costly than evaluation-free model selection.…”

Section: Evaluation-based Model Selectionmentioning

confidence: 99%

AutoGML: Fast Automatic Model Selection for Graph Machine Learning

Park¹,

Rossi²,

Ahmed³

et al. 2022

Preprint

View full text Add to dashboard Cite

Given a graph learning task, such as link prediction, on a new graph dataset, how can we automatically select the best method as well as its hyperparameters (collectively called a model)? Model selection for graph learning has been largely ad hoc. A typical approach has been to apply popular methods to new datasets, but this is often suboptimal. On the other hand, systematically comparing models on the new graph quickly becomes too costly, or even impractical. In this work, we develop the first meta-learning approach for automatic graph machine learning, called AUTOGML, which capitalizes on the prior performances of a large body of existing methods on benchmark graph datasets, and carries over this prior experience to automatically select an effective model to use for the new graph, without any model training or evaluations. To capture the similarity across graphs from different domains, we introduce specialized meta-graph features that quantify the structural characteristics of a graph. Then we design a meta-graph that represents the relations among models and graphs, and develop a graph meta-learner operating on the metagraph, which estimates the relevance of each model to different graphs. Through extensive experiments, we show that using AUTOGML to select a method for the new graph significantly outperforms consistently applying popular methods as well as several existing meta-learners, while being extremely fast at test time.Preprint. Under review.

show abstract

“…Subsequently, deep learning models, such as deep knowledge tracing (DKT) [8], were developed, which model a student's learning process as a recurrent neural network (RNN), significantly improving the prediction performance of the traditional Bayesian-based KT. With the development of graph neural networks (GNN) [9], GNN-based KT models [10,11], which use the natural graph structure existing in skills to model students' cognition, have attracted considerable attention. Although KT models have developed rapidly in recent years, limitations still exist.…”

Section: Introductionmentioning

confidence: 99%

APGKT: Exploiting Associative Path on Skills Graph for Knowledge Tracing

Zhang¹,

Bu²,

Liu³

et al. 2022

Preprint

View full text Add to dashboard Cite

Knowledge tracing (KT) is a fundamental task in educational data mining that mainly focuses on students' dynamic cognitive states of skills. The question-answering process of students can be regarded as a thinking process that considers the following two problems. One problem is which skills are needed to answer the question, and the other is how to use these skills in order. If a student wants to answer a question correctly, the student should not only master the set of skills involved in the question, but also think and obtain the associative path on the skills graph. The nodes in the associative path refer to the skills needed and the path shows the order of using them. The associative path is referred to as the skill mode. Thus, obtaining the skill modes is the key to answering questions successfully. However, most existing KT models only focus on a set of skills, without considering the skill modes. We propose a KT model, called APGKT, that exploits skill modes. Specifically, we extract the subgraph topology of the skills involved in the question and combine the difficulty level of the skills to obtain the skill modes via encoding; then, through multi-layer recurrent neural networks, we obtain a student's higher-order cognitive states of skills, which is used to predict the student's future answering performance. Experiments on five benchmark datasets validate the effectiveness of the proposed model.

show abstract

Automatic Graph Learning with Evolutionary Algorithms: An Experimental Study

Cited by 6 publications

References 19 publications

Evaluating the Performance of Automated Machine Learning (AutoML) Tools for Heart Disease Diagnosis and Prediction

Evaluating the Performance of Automated Machine Learning (AutoML) Tools for Heart Disease Diagnosis and Prediction

AutoGML: Fast Automatic Model Selection for Graph Machine Learning

APGKT: Exploiting Associative Path on Skills Graph for Knowledge Tracing

Contact Info

Product

Resources

About