Breiman's "Two Cultures'' Revisited and Reconciled

Mukhopadhyay, Deep; Wang, Kaijun

doi:10.36227/techrxiv.16569201.v1

Cited by 6 publications

(4 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In doing so, we moved the needle on an analytic framework that bridges the gap between the machine learning and traditional statistical “cultures” as they were referred to in Leo Breiman’s landmark paper (Breiman, 2001). Breiman’s paper, as well as numerous other articles (e.g., Donoho, 2017; Mukhopadhyay & Wang, 2020; Yarkoni & Westfall, 2017), have advocated for a blended culture so that the best method for a given task is chosen, such as LASSO for increasing the predictive ability of a model in future samples. The presence of missing data, a common and practical issue in psychological analyses, often supersedes conceptual reasons for selecting analytic approaches.…”

Section: Discussionmentioning

confidence: 99%

How to apply variable selection machine learning algorithms with multiply imputed data: A missing discussion.

Gunn¹,

Rezvan²,

Fernández³

et al. 2023

Psychological Methods

View full text Add to dashboard Cite

Psychological researchers often use standard linear regression to identify relevant predictors of an outcome of interest, but challenges emerge with incomplete data and growing numbers of candidate predictors. Regularization methods like the LASSO can reduce the risk of overfitting, increase model interpretability, and improve prediction in future samples; however, handling missing data when using regularization-based variable selection methods is complicated. Using listwise deletion or an ad hoc imputation strategy to deal with missing data when using regularization methods can lead to loss of precision, substantial bias, and a reduction in predictive ability. In this tutorial, we describe three approaches for fitting a LASSO when using multiple imputation to handle missing data and illustrate how to implement these approaches in practice with an applied example. We discuss implications of each approach and describe additional research that would help solidify recommendations for best practices.

show abstract

Section: Discussionmentioning

confidence: 99%

How to apply variable selection machine learning algorithms with multiply imputed data: A missing discussion.

Gunn¹,

Rezvan²,

Fernández³

et al. 2023

Psychological Methods

View full text Add to dashboard Cite

show abstract

“…A description of most machine learning models can be found in textbooks, such as those by Hastie et al (2009), while we consider that the reader is familiar with machine learning algorithms (e.g., with random forests or boosting algorithms) in the following. Transforming a machine learning model to issue probabilistic predictions is possible by combining relevant fundamental concepts introduced earlier (see also Mukhopadhyay and Wang 2020, for a relative view that focuses on the connections between parametric and non-parametric models).…”

Section: Machine Learning Algorithmsmentioning

confidence: 99%

A review of predictive uncertainty estimation with machine learning

Tyralis,

Papacharalampous

2024

Artif Intell Rev

View full text Add to dashboard Cite

Predictions and forecasts of machine learning models should take the form of probability distributions, aiming to increase the quantity of information communicated to end users. Although applications of probabilistic prediction and forecasting with machine learning models in academia and industry are becoming more frequent, related concepts and methods have not been formalized and structured under a holistic view of the entire field. Here, we review the topic of predictive uncertainty estimation with machine learning algorithms, as well as the related metrics (consistent scoring functions and proper scoring rules) for assessing probabilistic predictions. The review covers a time period spanning from the introduction of early statistical (linear regression and time series models, based on Bayesian statistics or quantile regression) to recent machine learning algorithms (including generalized additive models for location, scale and shape, random forests, boosting and deep learning algorithms) that are more flexible by nature. The review of the progress in the field, expedites our understanding on how to develop new algorithms tailored to users’ needs, since the latest advancements are based on some fundamental concepts applied to more complex algorithms. We conclude by classifying the material and discussing challenges that are becoming a hot topic of research.

show abstract

“…2. For more discussion on this topic, see Appendix A.6 and Mukhopadhyay and Wang (2020). In short, we have developed a process of constructing an admissible (explainable and efficient) ML procedure starting from a 'pure prediction' algorithm.…”

Section: Coreglm: Breast Cancer Wisconsin Datamentioning

confidence: 99%

InfoGram and admissible machine learning

Mukhopadhyay

2022

Mach Learn

View full text Add to dashboard Cite

We have entered a new era of machine learning (ML), where the most accurate algorithm with superior predictive power may not even be deployable, unless it is admissible under the regulatory constraints. This has led to great interest in developing fair, transparent and trustworthy ML methods. The purpose of this article is to introduce a new information-theoretic learning framework (admissible machine learning) and algorithmic risk-management tools (InfoGram, L-features, ALFA-testing) that can guide an analyst to redesign off-theshelf ML methods to be regulatory compliant, while maintaining good prediction accuracy. We have illustrated our approach using several real-data examples from financial sectors, biomedical research, marketing campaigns, and the criminal justice system.

show abstract

Breiman's "Two Cultures'' Revisited and Reconciled

Cited by 6 publications

References 32 publications

How to apply variable selection machine learning algorithms with multiply imputed data: A missing discussion.

How to apply variable selection machine learning algorithms with multiply imputed data: A missing discussion.

A review of predictive uncertainty estimation with machine learning

InfoGram and admissible machine learning

Contact Info

Product

Resources

About