An Introduction to Ensemble Methods for Data Analysis

Berk, Richard A.

doi:10.1177/0049124105283119

Cited by 125 publications

(119 citation statements)

References 25 publications

Supporting

Mentioning

109

Contrasting

Unclassified

Order By: Relevance

“…Our decision here to utilise random forest technique is informed by evidence in the social science literature that ensemble methods carry the potential for additional roles on top of the more common classification and forecasting roles, namely, serving as diagnostics tools to conventional methods, and providing further insight into the relation between the explanatory and response variables [39]. Moreover, their conceptual resemblance to wider adopted methods such as regression trees [27], make them a suitable candidate to introduce as a data-driven methodology into the process.…”

Section: Enhancing the Workflowmentioning

confidence: 99%

Supporting theoretically-grounded model building in the social sciences through interactive visualisation

et al. 2017

View full text Add to dashboard Cite

This is the accepted version of the paper.This version of the publication may differ from the final published version. Permanent repository link AbstractThe primary purpose for which statistical models are employed in the social sciences is to understand and explain phenomena occurring in the world around us. In order to be scientifically valid and actionable, the construction of such models need to be strongly informed by theory. To accomplish this, there is a need for methodologies that can enable scientists to utilise their domain knowledge effectively even in the absence of strong a priori hypotheses or whilst dealing with complex datasets containing hundreds of variables and leading to large numbers of potential models. In this paper, we describe enhanced model building processes in which we use interactive visualisations as the underlying mechanism to facilitate the construction and documentation of theory-driven models. We report our observations from a collaborative project involving social and computer scientists, and identify key roles for visualisation to support model building within the context of social science. We describe a suite of techniques to facilitate the exploration of statistical summaries of input variables, to compare the quality of alternative models, and to keep track of the model-building process. We demonstrate how these techniques operate in coordination to allow social scientists to efficiently generate models that are tightly underpinned by domain specific theory.

show abstract

Section: Enhancing the Workflowmentioning

confidence: 99%

Supporting theoretically-grounded model building in the social sciences through interactive visualisation

et al. 2017

View full text Add to dashboard Cite

show abstract

“…This tree can be used for prediction, by 'dropping' new observations down the tree (Breiman et al, 1984). For a more extensive description of the CART algorithm, see Berk (2006) or Strobl, Malley, and Tutz (2009). Gigerenzer and Goldstein (1996) and Gigerenzer et al (1999) have suggested CART as a powerful algorithm for the creation of simple decision making tools, because CART trees, like FFTs evaluate one cue at a time in order to arrive at a final decision.…”

Section: Classification and Regression Treesmentioning

confidence: 99%

“…RuleFit is a so-called ensemble method (e.g., Berk, 2006): it combines the predictions of multiple simple prediction functions to make a final prediction. The RuleFit model, as most learning ensembles, takes the form…”

Section: Rulefit Algorithmmentioning

confidence: 99%

Connecting clinical and actuarial prediction with rule-based methods.

Fokkema¹,

Smits²,

Kelderman³

et al. 2015

Psychological Assessment

View full text Add to dashboard Cite

Meta-analyses comparing the accuracy of clinical versus actuarial prediction have shown actuarial methods to outperform clinical methods, on average. However, actuarial methods are still not widely used in clinical practice, and there has been a call for the development of actuarial prediction methods for clinical practice. In this paper, we argue that rule-based methods may be more useful than the linear main effect models usually employed in prediction studies, from a data and decision analytic, as well as a practical perspective. In addition, decision rules derived with rule-based methods can be represented as fast and frugal trees, which, unlike main effects models, can be used in a sequential fashion, reducing the number of cues that have to be evaluated before making a prediction. We illustrate the usability of rule-based methods by applying RuleFit, an algorithm for deriving decision rules for classification and regression problems, to a dataset on prediction of the course of depressive and anxiety disorders from Penninx et al. (2011). The RuleFit algorithm provided a model consisting of two simple decision rules, requiring evaluation of only two to four cues. Predictive accuracy of the two-rule model was very similar to that of a logistic regression model incorporating 20 predictor variables, originally applied to the dataset. In addition, the two-rule model required evaluation of only three cues, on average. Therefore, the RuleFit algorithm appears to be a promising method for creating decision tools that are less time consuming and easier to apply in psychological practice, and with accuracy comparable to traditional actuarial methods.

show abstract

“…As noted there, ensemble learning focuses on the handling of the output of two or more intelligent systems in parallel. The reviews provided by [2,3] are recommended for their particular value in understanding ensemble methods.…”

Section: I E N S E M B L E a R C H I T E C T U R E Smentioning

confidence: 99%

The rationale for ensemble and meta-algorithmic architectures in signal and information processing

Simske¹

2015

SIP

View full text Add to dashboard Cite

We are living through an historic era in computing. As I . I N T R O D U C T I O NMachine intelligence is all around us. Our financial institutions sift through our purchasing data to define patterns of usual -and by their complement unusual -activity in order to minimize fraud. Most likely words and queries show up in typed, spoken, and gestured applications and services. Selfdriving cars are part of the everyday conversation. It is clear that a growing percentage of the intelligent decisions made in our world are performed by the machines, and not the minds, of humans.This "additional level of indirection" for decision-making comes at a price. There is the risk that reliance on machine algorithms may make humans less capable in some intellectual capacities; however, this issue will not be addressed here. What is more germane -and potentially more frightening -is that machines will make critical mistakes which go unnoticed. This argument is not based on an apocalyptic or Luddite fear of computers. Computers have arguably improved human lives more than any other invention (water purification and agriculture notwithstanding). Instead, it is because machine intelligence is so very different from human intelligence that the reliability of machine intelligence output must be questioned. Computers and their associated algorithms, systems and intelligence engines outperform humans on many mathematical and analytical tasks. To date, however, humans significantly outperform computers on more distributed, Gestalt, intuitive and unstructured tasks. As computers are tasked more and more in this purported "Big Data" era to discern information from ever-growing amounts of unstructured data, how can we be sure that the machine intelligence is consistent with human intelligence? One approach is to design machine intelligence systems that share the precepts of human intelligence. Paramount among these are the following precepts:• The ability to incorporate new rules of discernment and models for analysis into an existing model or set of models; • The ability to selectively apply different learned content based on the context; • The ability to correctly solve problems in the absence of sufficient input; • The ability to perform "real time ground truthing"; that is, the ability to assess a situation after a particular output is assumed and efficiently perform a minimal set of validation experiments to prove or disprove the correctness of the output.These precepts will be re-considered (Section VII) after the stage for meta-algorithm-based architectures is given. The ensemble and meta-algorithmic approaches and their

show abstract

An Introduction to Ensemble Methods for Data Analysis

Cited by 125 publications

References 25 publications

Supporting theoretically-grounded model building in the social sciences through interactive visualisation

Supporting theoretically-grounded model building in the social sciences through interactive visualisation

Connecting clinical and actuarial prediction with rule-based methods.

The rationale for ensemble and meta-algorithmic architectures in signal and information processing

Contact Info

Product

Resources

About