Energy modeling of Hoeffding tree ensembles

Abstract

Energy consumption reduction has been an increasing trend in machine learning over the past few years due to its socio-ecological importance. In new challenging areas such as edge computing, energy consumption and predictive accuracy are key variables during algorithm design and implementation. State-of-the-art ensemble stream mining algorithms are able to create highly accurate predictions at a substantial energy cost. This paper introduces the nmin adaptation method to ensembles of Hoeffding tree algorithms, to further reduce their energy consumption without sacrificing accuracy. We also present extensive theoretical energy models of such algorithms, detailing their energy patterns and how nmin adaptation affects their energy consumption. We have evaluated the energy efficiency and accuracy of the nmin adaptation method on five different ensembles of Hoeffding trees under 11 publicly available datasets. The results show that we are able to reduce the energy consumption significantly, by 21% on average, affecting accuracy by less than one percent on average.

Balancing Performance and Energy Consumption of Bagging Ensembles for the Classification of Data Streams in Edge Computing

“…In [9], the authors presented an energy-efficient approach to real-time prediction with high levels of accuracy called nmin adaptation, which reduces the energy consumption of Hoeffding Trees ensembles by adapting the number of instances required for a node split. This method can reduce energy consumption by 21% on average with a small impact on accuracy.…”

“…Finally, in essence, mini-batching consists of applying a loop-exchange optimization (by iterating over the ensembles in the outermost loop and then over the data instances in the innermost loop). As mini-batching does not change the implementation of the individual learners, it can wrap around other individual learners, along with their algorithm-specific optimizations (e.g., in [19], [9], [17], [16]. This, however, is out of this work's scope.…”

“…By taking the energy model from [9] as a base and using the iRD (i.e., equation 7), it is possible to extend the model to account for the mini-batching by the following equation:…”

“…Nevertheless, little effort has been made to reduce the energy consumption of data stream algorithms [9]. To address this gap, we have extended previous research [10], [11] to investigate how to optimize the classification of data streams with bagging ensembles concerning energy efficiency, time performance (i.e., delay and throughput), and predictive performance.…”

Enhancing Detection of Malicious URLs Using Boosting and Lexical Features

“…It relies on the fact that we frequently can choose an attribute that is optimal for splitting (from a small sample). The basic idea is derived from the Hoeffding bound [25].…”