On-line Random Forests

Abstract

Random Forests (RFs) are frequently used in many computer vision and machine learning applications. Their popularity is mainly driven by their high computational efficiency during both training and evaluation while achieving state-of-the-art results. However, in most applications RFs are used off-line. This limits their usability for many practical problems, for instance, when training data arrives sequentially or the underlying distribution is continuously changing.In this paper, we propose a novel on-line random forest algorithm. We combine ideas from on-line bagging, extremely randomized forests and propose an on-line decision tree growing procedure. Additionally, we add a temporal weighting scheme for adaptively discarding some trees based on their out-of-bag-error in given time intervals and consequently growing of new trees. The experiments on common machine learning data sets show that our algorithm converges to the performance of the off-line RF. Additionally, we conduct experiments for visual tracking, where we demonstrate real-time state-of-the-art performance on well-known scenarios and show good performance in case of occlusions and appearance changes where we outperform trackers based on on-line boosting. Finally, we demonstrate the usability of on-line RFs on the task of interactive real-time segmentation.

A Discriminative Distance Learning–Based CBIR Framework for Characterization of Indeterminate Liver Lesions

“…In particular, the online RF of Saffari et al [16] trains decision trees of fixed depth and has a fixed structure which does not change with the observation of new cases once the tree depth limit is reached. We address this issue in our online RF algorithm with -The use of primed off-line learning to speed up convergence to a reasonable accuracy.…”

“…Similar to Saffari et al [16] and different to the Hoeffding tree, a split is simply generated after observing a certain specified number of instances (40 is the default value normally leading to best performance). For each feature, Gaussian distribution is assumed and is tracked online, and a split threshold value which maximizes the Gini Index value is selected.…”

“…In order to speed up our extensive experiments, make it possible to validate more trends and make the models adaptive to learn from new samples we implement an incrementalization of RF similar to Saffari [16]. While successful lossless incrementalizations exist for many learning algorithms, most strong techniques applied in real applications are still difficult to incrementalize, and among them perhaps the most prominent example that received considerable attention lately is the ensemble of randomized decision trees, the most famous representative of which is the Random Forest algorithm [15].…”

A Robust Multi-Target Visual Tracking Method using TLD Framework

“…The author compared TLD with some other algorithms, like IVT (Iterative Visual Tracking [8]), ODV (Online Discriminative Features [9]), ET (Ensemble Tracking [10]), MIL (Multiple Instance Learning [11]), Co-trained Generative Discriminative Tracking [12], OB (Online Boosting [13]), ORF (Online Random Forests [14]), PROST (Parallel Robust Online Simple Tracking [15]) etc. The performance is evaluated using P, R and F. P represents the number of true positives divided by the number of all responses and R represents the number of true positives divided by the number of object occurrences that should have been detected.…”

Hierarchy of Localized Random Forests for Video Annotation

“…Usually a bounding box is propagated to later frames. Boosting based classifiers [17] and random forest classifiers [16] are popular choices for learning and updating the template model. Godec et al [10] also give a rough segmentation of the object being tracked.…”