Color blindness can be challenging as well as cumbersome for a person especially when driving a vehicle. In this paper, a real time traffic light recognition system based on computer vision will be described that improves the safety of drivers who are color-blind. The system consists of a digital video camera and a portable PC that helps the system to detect and track red, green, and yellow traffic lights. The paper will explain how the RGB component adjustment, thresholding algorithm, and median filter are used to recognize traffic lights. In order for the system to handle illumination changes by weather and time, a simple adaptation method has been developed. For future use, the portable system has been simplified since audio has been added to the current system, which helps to inform the traffic light the driver is color-blind.