In this paper, a new block-based motion estimation (ME) method is proposed which uses the Kalman filtering (KF) with adaptive block partitioning (ABP) to improve the motion estimates resulting from conventional blockmatching algorithms (BMAs). In our method, a first order autoregressive model is applied to the motion vectors (MVs) obtained by BMAs. The motion correlations between neighboring blocks are utilized to predict motion information. According to the statistics of the frame MVs, 16x16 macro-blocks (MBs) are split into 8x8 blocks or 4x4 sub-blocks adaptively for the Kalman filtering (KF). To further improve the performance, a zigzag scanning is adopted and the state parameters of the Kalman filter are adjusted adaptively during the each KF iteration. The experimental results indicate that the proposed method can effectively improve the ME performance in terms of the peak-signal-to-noise-ratio (PSNR) of the motion compensated images with smoother motion vector fields.