HiPaR: Hierarchical Pattern-Aided Regression

Comparing Machine-Learning Models of Different Levels of Complexity for Crop Protection: A Look into the Complexity-Accuracy Tradeoff

“…The local models are usually interpretable functions, e.g., linear functions, that capture local relationships between the target and the predictive variables that cannot be observed at the "global level". As shown in the literature [10,15], these methods exhibit higher predictive performance than linear regression at the price of a manageable increase in complexity. Examples of pattern-aided regression methods include piecewise regression [23], regression trees [20], model trees [24], 2 Contrast pattern-aided regression (CPXR) [15], and HiPaR [10].…”

“…A simple, yet effective proxy to model interpretability is model complexity [10]. Complexity is usually measured as the number of relevant parameters that play a role in the model's answers, and it is known to be correlated with interpretability.…”

“…While complex models such as neural networks or gradient boosting tend to be less interpretable than transparent simple methods such as linear regression or shallow trees, in some cases this complexity pays off in terms of prediction performance [11,12,10]. 1 This trade-off between complexity and prediction accuracy can happen because more parameters or weaker assumptions endow models with more expressiveness and flexibility to capture subtle interactions.…”

“…This, for instance, excludes any potential interactions between the input features as predictors for the target variable. Between complex approaches and simple models lie pattern-based models [10,15] that strike an interesting trade-off because they remain relatively simple white boxes that exhibit higher predictive power than linear regression or decision trees.…”

“…box ensemble methods such as random forests and gradient boosting trees; (ii) white boxes such as linear regression; and (iii) HiPaR [10], a pattern-based regression method of intermediate complexity. Our experiments confirm a clear complexity-accuracy trade-off in our use cases, and also show different techniques to distill agronomical insights from both white-and black-box ML models.…”