Intelligent vehicles require the ability to switch freely between manual and automatic driving modes, which opens up more possibilities for steer-by-wire. Among them, the design of transmission ratio for both driving modes can improve the steering performance of vehicle. In the manual driving mode, this paper optimizes the yaw rate gain upon the particle swarm optimization (PSO) algorithm and designs a variable steering transmission ratio, which can reduce the driver’s burden and improve the vehicle’s handling stability. Further, the steering wheel angular rate information and fuzzy control are used to design the high-speed transmission ratio to ensure the safety of the vehicle in high-speed emergency situations. In the automatic driving mode, the transmission ratio values at various vehicle speeds are optimized offline based on the PSO algorithm, and a variable transmission ratio curve is designed to enhance the trajectory tracking ability and the vehicle stability. Finally, the hardware-in-the-loop test results show that the variable transmission ratio designed in the manual driving mode reduces the burden of the driver at low speed and improves the vehicle stability at high speed, especially in emergency situations. It also proves that the optimized design of variable ratio in the automatic driving mode promotes the path-following ability and the vehicle stability.