Naturally occurring asbestos (NOA) represents a matter of social and environmental concern due to its potential release in the atmosphere during rock excavation and grinding in quarry and road tunnel activities. In most cases, NOA occurs in serpentinites, i.e., rocks deriving from low-grade metamorphic hydration of mantle peridotites. The potential release of asbestos fibers from serpentinite outcrops depends on several features, such as serpentinization degree, rock deformation, weathering, and abundance of fibrous veins. In this study, we selected a set of serpentinite samples from a representative outcrop in Tuscany (Italy), and we analyzed them by Optical, Scanning, and Transmission Electron Microscopies. The samples were treated by grinding tests following the Italian guidelines Decrees 14/5/96 and 152/2006 for the determination of the Release Index (RI), i.e., the fiber amount released through controlled crushing tests. The fine-grained powder released during the tests was analyzed by quantitative Fourier transform infrared spectroscopy (FTIR) to determine the variety and the amount of released fibers and to assess the potential hazard of the different serpentinite samples. Results indicate that the amount of released fibers is mostly related to serpentinite deformation, with the highest RI values for cataclastic and foliated samples, typically characterized by widespread occurrence of fibrous veins. Conversely, massive pseudomorphic serpentinite revealed a very low RI, even if their actual chrysotile content is up to 20–25%. Based on our original findings from the RI results, a preliminary investigation of the outcrop at the mesoscale would be of primary importance to obtain a reliable hazard assessment of NOA sites, allowing the primary distinction among the different serpentinites lithotypes and the effective fiber release.