In recent years, understanding human behavior has become one of the most important topics in the field of computer vision research. The reason for this growing attention is the wide range of applications that can benefit from the results of this field of research. The human behavior analysis (HBA) includes a wide range of research areas from the detection of human motion and action.Datasets created through actions and detection of human activities make it possible to compare different detection methods with the same input data. Data mining and big data approaches are very popular for analyzing data related to human behavior and can be used to address the challenges of fast processing. This article provides a systematic survey of data mining and big data in the HBA. We focus on current datasets and models related to the detection of human behavior patterns in the literature. The purpose of this survey is to assist researchers in select appropriate datasets and models to evaluate algorithms, as well as to identify research gaps for future work. To achieve this goal, articles published between 2010 and 2021 have been reviewed. These articles fall into five general categories in terms of dataset focus: object detection, motion, action, activity, and behavior. This article provides a summary of data mining and big data models in the HBA, as well as related datasets based on these categories, to highlight promising research avenues for future work.