Time synchronization (TS) is crucial for ensuring the secure and reliable functioning of the distribution power Internet of Things (IoT). Multi-clock source time synchronization (MTS) has significant advantages of high reliability and accuracy but still faces challenges such as optimization of the multi-clock source selection and the clock source weight calculation at different timescales, and the coupling of synchronization latency jitter and pulse phase difference. In this paper, the multi-timescale MTS model is conducted, and the reinforcement learning (RL) and analytic hierarchy process (AHP)-based multi-timescale MTS algorithm is designed to improve the weighted summation of synchronization latency jitter standard deviation and average pulse phase difference. Specifically, the multi-clock source selection is optimized based on Softmax in the large timescale, and the clock source weight calculation is optimized based on lower confidence bound-assisted AHP in the small timescale. Simulation shows that the proposed algorithm can effectively reduce time synchronization delay standard deviation and average pulse phase difference.