Machine Learning for Multi-Output Regression: When should a holistic multivariate approach be preferred over separate univariate ones?

Abstract

Tree-based ensembles such as the Random Forest are modern classics among statistical learning methods. In particular, they are used for predicting univariate responses. In case of multiple outputs the question arises whether we separately fit univariate models or directly follow a multivariate approach. For the latter, several possibilities exist that are, e.g. based on modified splitting or stopping rules for multi-output regression. In this work we compare these methods in extensive simulations to help in answering the primary question when to use multivariate ensemble techniques.

Supervised Machine-Learning Algorithm using Low Data Sets: Flow Chemistry Optimization of the Key Urea Moiety Construction in Larotrectinib

“…In the current experimental process involving multiple parameter optimization, supervised ML was employed. Given the insights into feature selection and correlation, a single-output regression model seemed insufficient, leading us to choose multi-output (M/O) regression models . We initially utilized ensemble models such as bagging, boosting, and regularization algorithms for bias-variance trade-off.…”

The Role of Data Analytics in the Assessment of Pathological Speech—A Critical Appraisal

“…Pathological speech and voice evaluation approaches reviewed in the present study (Selected accordingly to the criteria before mentioned: addressing laryngeal pathology characterization from acoustical signal analysis, having been published since 2010, disclosing classification or regression methodology, based on standard databases or supplementary materials to allow reproducibility, supported by reliable performance metrics, and contributing novel and original work.). EPAS: Exploratory or prospective advanced statistics; SVP: Statistical validation of performance; DD/MD: Data-driven or model-driven (the label within parenthesis indicates the strategy of the approach regarding Figure 3, as a, b, c, d difficult to classify and even talk about a multi-perspective problem when it is not visualized, not to mention interpretability and explainability, bearing in mind that many of these disorders induce similar observable correlates (many-to-many regression and classification problems [11]). such as Gaussian mixture models, random forests, shallow neural networks, support vector machines, etc.…”

“…The critical question is that this polyhedral panorama might sometimes be ignored by publications in machine-learning laryngeal pathology characterization, although they strongly condition the behavior of the biomechanical system and the acoustical signals, which is the ground on which pathology detection and classification stand. It is really difficult to classify and even talk about a multi-perspective problem when it is not visualized, not to mention interpretability and explainability, bearing in mind that many of these disorders induce similar observable correlates (many-to-many regression and classification problems [11]).…”