The Buddhist sacred site of Taya Caves is a gem hidden underground in Yokohama, Japan. The caves were excavated and sculpted into bare rock by Shingon Buddhist monks from the Kamakura until the Edo period (thirteenth-nineteenth century), and dedicated to ascetic training, rituals, and pilgrimage. They are a maze of halls and galleries decorated with hundreds of rock-cut reliefs, picturing deities and masters of Buddhism, temples and shrines, real and fantastic animals, vegetal motifs, mandalas, zodiac signs, family crests, etc. The history and rock art of Taya Caves and the urge to preserve their cultural value led to this first-ever scientific investigation, dealing with the stone properties, deterioration, and environmental setting. Textural, mineralogical, geochemical, and petrophysical investigations were combined with a microclimate monitoring and chemical analyses of groundwater and rainwater. The caves are excavated into a clay-rich fossiliferous siltstone, extremely soft and porous and highly susceptible to water-driven weathering. Water represents a constant in Taya Caves, either flowing, dripping, and stagnant; or rising from the subsoil; or related to the extremely high relative humidity. Crusts and efflorescences represent important indicators of mineral dissolution and mobilization. The crusts are made of gypsum, crystallized from the dissolution of calcareous bioclasts and oxidation of pyrite, with minor calcite. The efflorescences are composed of chlorides, phosphates, sulfates, and carbonates, possibly deriving from agrochemicals and the surface vegetation cover. The salt weathering is strictly related to the microenvironmental variables and physico-chemical properties of the phases and waters involved. Rock-water interaction is particularly damaging even considering just the physical mechanisms. The stone is strongly sensitive to water absorption, hygroscopic adsorption, and slaking: the stresses generated by in-pore water and air movement and the swelling clay minerals may lead to rapid disintegration, especially during cyclic processes. This research is expected to raise concerns about the safeguard of Taya Caves and support future monitoring and conservation plans, and to foster a wider promotion and valorization of this heritage site.