Computer clusters with the shared-nothing architecture are the major computing platforms for big data processing and analysis. In cluster computing, data partitioning and sampling are two fundamental strategies to speed up the computation of big data and increase scalability. In this paper, we present a comprehensive survey of the methods and techniques of data partitioning and sampling with respect to big data processing and analysis.We start with an overview of the mainstream big data frameworks on Hadoop clusters. The basic methods of data partitioning are then discussed including three classical horizontal partitioning schemes: range, hash, and random partitioning. Data partitioning on Hadoop clusters is also discussed with a summary of new strategies for big data partitioning, including the new Random Sample Partition (RSP) distributed model. The classical methods of data sampling are then investigated, including simple random sampling, stratified sampling, and reservoir sampling. Two common methods of big data sampling on computing clusters are also discussed: record-level sampling and blocklevel sampling. Record-level sampling is not as efficient as block-level sampling on big distributed data. On the other hand, block-level sampling on data blocks generated with the classical data partitioning methods does not necessarily produce good representative samples for approximate computing of big data. In this survey, we also summarize the prevailing strategies and related work on sampling-based approximation on Hadoop clusters. We believe that data partitioning and sampling should be considered together to build approximate cluster computing frameworks that are reliable in both the computational and statistical respects.
Neuro-fuzzy system is now one of the most widely used tools in the field of artificial intelligence systems. This study proposes a novel approach for time series stock market price prediction using a recurrent error-based neuro-fuzzy system with momentum (RENFSM). The basic idea of this approach is to use time series price momentum and time series prediction error adjusted to the well-known adaptive neuro-fuzzy inference system, ANFIS. Extended from ANFIS, the aim of this study is to propose a reliable prediction system with minimal error. Moreover, to evaluate the proposed model strength, four top-listed stocks from Dhaka stock exchange were applied. In the experiments, several choices of momentum from 3 to 20 days are selected for data preprocessing. It was found that the proposed RENFSM performed superiorly and was more reliable compared to the existing methods such as ANFIS and neural networks.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.