Towards Meta-Learning for Multi-Target Regression Problems

Aguiar, Gabriel Jonas; Santana, Everton José; Mastelini, Saulo Martiello; Mantovani, Rafael Gomes; Barbon, Sylvio

doi:10.1109/bracis.2019.00073

Cited by 5 publications

(6 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Spyromitros-Xioufis et al ( 2016) introduced the stacked ST (SST) and ensemble RC (ERC). These methods can be computationally complex with high memory costs (Mastelini et al 2019). As Aguiar et al (2019) state, choosing the most suitable approach needs previous testing and depends on the task.…”

Section: ) Multi-step Approachmentioning

confidence: 99%

See 1 more Smart Citation

A Geostationary Lightning Pseudo-Observation Generator Utilizing Low-Frequency Ground-Based Lightning Observations

Erdmann,

Caumont,

Defer

2022

Journal of Atmospheric and Oceanic Technology

View full text Add to dashboard Cite

Coincident Geostationary Lightning Mapper (GLM) and National Lightning Detection Network (NLDN) observations are used to build a generator of realistic lightning optical signal in the perspective to simulate Lightning Imager (LI) signal from European NLDN-like observations. Characteristics of GLM and NLDN flashes are used to train different machine learning (ML) models, that predict simulated pseudo-GLM flash extent, flash duration, and event number per flash (targets) from several NLDN flash characteristics. Comparing statistics of observed GLM targets and simulated pseudo-GLM targets, the most suitable ML-based target generators are identified. The simulated targets are then further processed to obtain pseudo-GLM events and flashes. In the perspective of lightning data assimilation, Flash Extent Density (FED) is derived from both observed and simulated GLM data. The best generators simulate accumulated hourly FED sums with a bias of 2% to the observation, while cumulated absolute differences remain of about 22 %. A visual comparison reveals that hourly simulated FED features local maxima at the similar geolocations as the FED derived from GLM observations. However, the simulated FED often exceeds the observed FED in regions of convective cores and high flash rates. The accumulated hourly area with FED>0 flashes per 5 km×5 km pixel simulated by some pseudo-GLM generators differs by only 7% to 8% from the observed values. The recommended generator uses a linear Support Vector Regressor (linSVR) to create pseudo-GLM FED. It provides the best balance between target simulation, hourly FED sum, and hourly electrified area.

show abstract

Section: ) Multi-step Approachmentioning

confidence: 99%

“…These methods can be computationally complex with high memory costs (Mastelini et al 2019). As Aguiar et al (2019) state, choosing the most suitable approach needs previous testing and depends on the task. The methods cited here are computationally expensive.…”

Section: ) Multi-step Approachmentioning

confidence: 99%

A Geostationary Lightning Pseudo-Observation Generator Utilizing Low-Frequency Ground-Based Lightning Observations

Erdmann,

Caumont,

Defer

2022

Journal of Atmospheric and Oceanic Technology

View full text Add to dashboard Cite

show abstract

“…They are directly extracted from data with significant computational costs using methods with significant complexity but without hyperparameters, requiring significant amounts of data. They are based on discrete attributes and include properties such as entropy which captures the amount of information and complexity of data, mutual information which mostly determines the relation of attributes and target class used for classification problems [25]. They are used for presenting different behavior patterns [29], for performing high quality recommendations, and for representing inner correlations between different classes [30].…”

Section: Meta-featuresmentioning

confidence: 99%

“…So far, domain specific meta-features are not used in the anomaly detection domain. They are only used in text classification, where domain-based knowledge present vocabulary length, words overlap, number of text categories, corpus hardness, domain broadness, and similar [25], [31].…”

Section: Meta-featuresmentioning

confidence: 99%

Novel Meta-Features for Automated Machine Learning Model Selection in Anomaly Detection

et al. 2021

View full text Add to dashboard Cite

A growing number of research papers shed light on automated machine learning (AutoML) frameworks, which are becoming a promising solution for building complex machine learning models without human expertise and assistance. The key challenge in enabling AutoML frameworks to build an efficient model for anomaly detection tasks is to determine the best underlying model for a given task and optimization metric. The meta-learning approaches based on a set of meta-features that describes data properties can enable efficient model selection in AutoML frameworks. The existing meta-learning approaches based on statistical and information-theoretic meta-features require large amounts of data and computational resources to extract data properties. This paper proposes a novel set of meta-features for model selection in anomaly detection tasks based on domain-specific properties of data which overcomes the shortcomings of existing meta-features by introducing simple but effective meta-features that can be efficiently extracted or estimated by using a low amount of data. Experiments with 63 datasets from different repositories with varying schemas show that the proposed set of meta-features achieves an accuracy of 87% for model selection, while the achieved accuracy for simple meta-features is 74%, for statistical meta-features 68%, for information theory meta-feature 70%, and for a comprehensive set of meta-features by pyMFE 73%. This demonstrates that the proposed set can be adopted by AutoML frameworks across a diverse range of domains.

show abstract

“…Another potential solution is to tackle the problem through a divide-and-conquer paradigm, since no single ML algorithm can achieve full coverage over all possible contexts, discrepancies and noises. This is also called the 'no free lunch' theorem [8], [9]. Instead, one could design different ML models where each performs sufficient enough in one context, but lacks performance in others.…”

Section: Introductionmentioning

confidence: 99%

Metamodelling of Noise to Image Classification Performance

et al. 2023

View full text Add to dashboard Cite

Machine Learning (ML) has made its way into a wide variety of advanced applications, where high accuracies can be achieved when these ML models are evaluated in the same context as they were trained and validated on. However, when these high-accuracy models are exposed to out-of-distribution points such as noisy inputs, their performance could potentially degrade significantly. Recommending the most suitable ML model that retains a higher accuracy when exposed to these noisy inputs can overcome this performance degradation. For this, a mapping between the noise distribution at the input and the resulting accuracy needs to be obtained. Though, this relationship is costly to evaluate as this is a computationally intensive task. To minimize this computational cost, we employ metalearning to predict this mapping; that is, the performance of different ML models is predicted given the distribution parameters of the input noise. Although metalearning is an established research field, performance predictions based on noise distribution parameters have not been accomplished before. Hence, this research focuses on predicting the per-class classification performance based on the distribution parameters of the input noise. For this, our approach is twofold. First, in order to gain insights in this noise-to-performance relationship, we analyse the per-class performance of well-established convolutional neural networks through our multi-level Monte Carlo simulation. Second, we employ metalearning to learn this relationship between the input noise distribution and the resulting per-class performance in a sample-efficient way by incorporating Latin Hypercube Sampling. The noise performance analyses present novel insights about the per-class performance degradation when gradually increasing noise is augmented on the input. Additionally, we show that metalearning is capable of accurately predicting the per-class performance based on the noise distribution parameters. We also show the relationship between the number of metasamples and the metaprediction accuracy. Consequently, this research enables future work to make accurate classifier recommendations in noisy environments.

show abstract

Towards Meta-Learning for Multi-Target Regression Problems

Cited by 5 publications

References 29 publications

A Geostationary Lightning Pseudo-Observation Generator Utilizing Low-Frequency Ground-Based Lightning Observations

A Geostationary Lightning Pseudo-Observation Generator Utilizing Low-Frequency Ground-Based Lightning Observations

Novel Meta-Features for Automated Machine Learning Model Selection in Anomaly Detection

Metamodelling of Noise to Image Classification Performance

Contact Info

Product

Resources

About