End-to-End Temporal Action Detection Using Bag of Discriminant Snippets

Vectors of temporally correlated snippets for temporal action detection

“…Framework FPS Sparse-prop [24] Two-stage 10.2 SCNN [4] Multi-stage 60.0 DAPS [14] Two-stage 134.3 CDC [7] Single-stage 500.0 TURN [8] Two-stage 880.8 R-C3D [17] Two-stage 1030.0 BoDS [3] Single-stage 1279.0 VTCS (Ours) Two-stage 1141.0…”

“…Our previous work i.e. BoDS [3] was based upon Bag of Words (BoW) but with the difference that the discriminative power of the key-snippets (centroids) was integrated during the encoding process in terms of discriminative weights. BoDS encoding for multiple snippets is calculated based upon the sum of weighted histograms, where the weight for the specific cluster is calculated by computing the ratio between the within- class assignments and the total assignments for that cluster.…”

“…To obtain the action proposals from the untrimmed videos, a sliding-window strategy is adopted of length equal to τ snippets from [3] to generate temporal proposals of multiple durations. At each time step t, the window is slid while producing left-aligned proposals set P t = {p h } τ h=1 .…”

“…it is about 1.3 times faster than TURN [8] and 1.1 times faster than R-C3D [17] which also use C3D features to represent video units. BoDS [3] was tested on an Nvidia TITAN Xp GPU and SCNN [4] on a GTX 980 GPU. DAPS [14], R-C3D [17], TURN [8] and CDC [7] were tested on a single Nvidia TITAN X GPU.…”

“…Hence, these methods fail to distinguish an action of interest from other actions of interest and from the background class. In our previous work, we proposed the Bag of Discriminant Snippets (BoDS) [3] approach. BoDS utilizes the discriminant importance of key-snippets in the encoding process by learning the weights of key-snippets.…”