An Enhanced Human Activity Prediction at Rainy and Night Times

Abstract

Human activity prediction aims to recognize an unfinished activity with limited motion and appearance information. A generalized activity prediction framework was proposed for human activity prediction where Probabilistic Suffix Tree (PST) was introduced to model casual relationships between constituent actions. Then, each kind of activity in videos was predicted by modeling interactive object information through Spatial Pattern Mining (SPM). This framework mined the temporal sequence patterns. For efficient human activity prediction a Spatio-Temporal Frequent Object Mining (STOM) was proposed in which the spatial, size and motion correlation among objects information were collected along with the temporal information. After the collection of this information, the objects were identified by using Modified Histogram Of Gradient (MHOG) and then the objects were tracked by particle filter technique. The frequent action of detected objects were identified by using Frequent Pattern-growth (FP-growth) which predicted the infrequent action as abnormal human activity in videos. However, MHOG based Object Detection and Tracking-STOM (MHOGODT-STFOM) based human activity prediction is not more effective at night time and rainy time. So in this paper, Enhanced Object Detection and Tracking- STFOM (EODTSTFOM) and Removing Rain Streaks-EODT-STFOM (RSREODT-STFOM) are proposed for human activity prediction even at night time and rainy time. In EODT, a modified Contrast Model is used which combined the contrast information and local entropy information to detect object contents present in the current image frame. Then, the objects are tracked by Kalman filter. In RSR-EODT, the rain streaks in the images are removed based on the deep Convolutional Neural Network (CNN). Then the objects are detected and tracked by modified Contrast Model and Kalman filter respectively. After the object detection and object tracking by EODT and RSR-EODT, the frequent actions are obtained by applying STFOM. The frequent actions are considered as normal activities and the infrequent actions are considered as abnormal activities. Thus the proposed EODTSTFOM and RSR-EODT- STFOM methods predict the human activities even at night time and rainy time.