The production of high-quality virgin olive oil from traditional olive (Olea europaea L.) varieties with peculiar and differential characteristics is of great interest for the olive oil market. ‘Castellana’ is an autochthonous variety mainly located in the center of Spain. The aims of this study were 1) the characterization of ‘Castellana’ virgin olive oils and 2) the evaluation of the influence of fruit ripening degree on the oil quality to establish an optimum harvest time for ‘Castellana’ olives. A wide range of physicochemical and sensorial quality parameters were assayed in oils produced at four harvest times during three crop seasons. ‘Castellana’ oils could be classified into the extra virgin category at all ripening degrees studied. This variety provides well-balanced oils from the sensorial point of view with an optimum chemical composition. Nevertheless, fruit maturation had a strong effect in various quality parameters, especially total phenol content, total tocopherol content, sensorial quality, and to a lesser extent in fatty acid composition. Loss of antioxidants and decrease in sensorial quality take place during olive ripening, reducing the nutritional, sensorial, and commercial quality of virgin olives oils as the harvest is delayed. Results suggest that the production of optimal extra virgin olive oil requires that ‘Castellana’ olives should be harvested from the middle of November to the middle of December, coinciding with a ripening index between 3.1 and 4.1. These results are of great importance to the olive oil industry for improving the quality of virgin olive oils produced from ‘Castellana’.