Raw bovine milk is highly nutritious as well as pH-neutral, providing the ideal conditions for microbial growth. The microbiota of raw milk is diverse and originates from several sources of contamination including the external udder surface, milking equipment, air, water, feed, grass, feces, and soil. Many bacterial and fungal species can be found in raw milk. The autochthonous microbiota of raw milk immediately after milking generally comprises lactic acid bacteria such as Lactococcus, Lactobacillus, Streptococcus, and Leuconostoc species, which are technologically important for the dairy industry, although they do occasionally cause spoilage of dairy products. Differences in milking practices and storage conditions on each continent, country and region result in variable microbial population structures in raw milk. Raw milk is usually stored at cold temperatures, e.g., about 4°C before processing to reduce the growth of most bacteria. However, psychrotrophic bacteria can proliferate and contribute to spoilage of ultra-high temperature (UHT) treated and sterilized milk and other dairy products with a long shelf life due to their ability to produce extracellular heat resistant enzymes such as peptidases and lipases. Worldwide, species of Pseudomonas, with the ability to produce these spoilage enzymes, are the most common contaminants isolated from cold raw milk although other genera such as Serratia are also reported as important milk spoilers, while for others more research is needed on the heat resistance of the spoilage enzymes produced. The residual activity of extracellular enzymes after high heat treatment may lead to technological problems (off flavors, physico-chemical instability) during the shelf life of milk and dairy products. This review covers the contamination patterns of cold raw milk in several parts of the world, the growth potential of psychrotrophic bacteria, their ability to produce extracellular heat-resistant enzymes and the consequences for dairy products with a long shelf life. This problem is of increasing importance because of the large worldwide trade in fluid milk and milk powder.