Detecting the onset of structural damage and its progressive evolution is crucial for the assessment and maintenance of the built environment. This paper describes the application of a computer-vision-based methodology for structural health monitoring to a shake table investigation. Three rubble stone masonry walls, one unreinforced and two reinforced, were tested under natural earthquake base inputs, progressively scaled up to collapse. White noise signals were also applied for dynamic identification purposes. Throughout the experiments, videos were recorded, under both white noise excitation and environmental vibrations, with the table at rest. The videos were preprocessed with motion magnification algorithms and analyzed through a principal component analysis. The natural frequencies of the walls were detected and their progressive decay was associated with damage accumulation. Results agreed with those obtained from another measurement system available in the laboratory and were consistent with the crack pattern development surveyed during the tests. The proposed approach proved useful to derive information on the progressive deterioration of the structural properties, showing the feasibility of this methodology for real field applications.