Landslide displacement prediction is an essential component for developing landslide early warning systems. In the Three Gorges Reservoir area (TGRA), landslides experience step-like deformations (i.e. periods of stability interrupted by abrupt accelerations) generally from April to September due to the influence of precipitation and reservoir scheduled level variations. With respect to many traditional machine learning techniques, two issues exist relative to displacement prediction, namely the random fluctuation of prediction results and inaccurate prediction when step-like deformations take place. In this study, a novel and original prediction method was proposed by combining the Wavelet Transform (WT) and Particle Swarm Optimization-Kernel Extreme Learning Machine (PSO-KELM) methods, and considering the landslide causal factors. A typical landslide with a step-like behavior, the Baishuihe landslide in TGRA, was taken as a case study. The cumulated total displacement was decomposed into trend displacement, periodic displacement (controlled by internal geological conditions and external triggering factors respectively), and noise. The displacement items were predicted separately by multi-factor PSO-KELM considering various causal factors, and the total displacement was obtained by summing them up. An accurate prediction was achieved by the proposed method, including the step-like deformation period. The performance of the proposed method was compared with the multi-factor Extreme Learning Machine (ELM), Support Vector Regression (SVR), Backward Propagation Neural Network (BPNN), and single-factor PSO-KELM. Results show that the PSO-KELM outperforms other models, and the prediction accuracy can be improved by considering causal factors.