This research delves deeply into the traffic light intersection control issue for connected autonomous vehicles (CAVs). Firstly, we employed the vehicle dynamics simulation software, CarSim, to model the vehicle, and utilized the intelligent driving simulation software, PreScan, to establish a road environment model. Subsequently, we designed a rule-based traffic light crossing controller (RB-TLCC) for CAVs in Matlab/Simulink and implemented a co-simulation with CarSim and PreScan. Furthermore, we conducted driver-in-the-loop studies using the Logitech G27 driving simulator kit and compared the experimental results with those from RB-TLCC. Our findings indicate that the RB-TLCC enhances regenerative braking energy by 23.5%, improves traffic efficiency by 17.9%, and increases driving smoothness by 50.7%. Additionally, based on Markov Chain theory, we proposed a multi-objective optimization model (MO-OM) for CAVs traffic light intersection crossing using the Proximal Policy Optimization algorithm (PPO). A comparison was conducted under two different operating conditions between RB-TLCC and dynamic programming algorithms (DP). The results indicate that the MO-OM proposed in this paper exhibits the best comprehensive control performance among the three control methods. While adhering to traffic regulations, it enhances the recuperation of braking energy, the efficiency of vehicle passage, and the smoothness of travel at traffic light intersections. This study offers effective methodologies for enhancing the performance of CAVs in traffic light intersection control and holds significant reference value for the advancement of future intelligent transportation systems (ITS).INDEX TERMS Connected autonomous vehicles, intelligent traffic system, multi-objective optimization, optimal speed planning, PPO algorithm, traffic light intersection control.