Since the number of fires in the world is rising rapidly, automatic fire detection is getting more and more interest in computer vision community. Instead of the usual inefficient sensors, captured videos by video surveillance cameras can be analyzed to quickly detect fires and prevent damages. This paper presents an early fire-alarm raising method based on image processing. The developed work is able to discriminate fire and non-fire pixels. Fire pixels are identified thanks to a rule-based color model built in the PJF color space. PJF is a newly designed color space that enables to better reflect the structure of the colors. The rules of the model are established through examining the color nature of fire. The proposed fire color model is assessed over the largest dataset in the literature collected by the authors and composed of diverse fire images and videos. While considering color information only, the experimental findings of detecting flame pixels candidates are promising. The suggested method achieves up to 99.8% fire detection rate and 8.59% error rate. A comparison with the state-of-the-art color models in different color spaces is also carried out to prove the performance of the model. Based on the color descriptor, the developed approach can accurately detect fire areas in the scenes and accomplish the best compromise between true and false detection rates.