3D MIMO introduces the vertical dimension of the antenna downtilt angle to make the direction of signal transmission more accurate to improve system capacity. In this paper, we verify the effect of antenna downtilt angle on channel capacity through simulations of four fixed antenna downtilt angles, 90, 96, 99, and 102 degrees under the conditions that the distance between mobile station (MS) and base station (BS) is 250 m, and the heights of antenna in BS and MS are 25 m and 1.5 m, respectively. The simulation results show that the antenna downtilt angle of 96 degrees has a larger channel capacity than the others. In addition, we proposed an adaptive optimization method by applying the Q-learning algorithm to adaptively optimize the antenna downtilt angles to maximize system capacity. The performance of the proposed method is to investigate the Q-learning algorithm with three different discount rates at 0.9, 0.5, and 0.1, and four different propagation distances on 20 × 1 and 60 × 4 MIMO. We demonstrate that there is only a 1% difference between the adaptively optimized antenna downtilt angle and the ideal optimal antenna downtilt angle when the discount rate of Q-learning algorithm is 0.9, and its channel capacity performance can reach more than 99.72% of the ideal optimal one.