Gaussian mixture model based phase prior learning for video motion estimation

Cai, Enjian; Zhang, Yi

doi:10.1016/j.ymssp.2022.109103

Cited by 13 publications

(6 citation statements)

References 46 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The second problem that should be concerned is that the phase estimation is prone to the ill‐posed problem (Cai & Zhang, 2022; Cai et al., 2022; Wadhwa et al., 2013), in which the observation noise and computational errors will be amplified, propagated, generating uncertainties in phase variations and eventually leading to incorrect structural motions. Therefore, estimating noisy structural motions is not easy.…”

Section: Sparsity‐enforcement Structural Motion Estimationmentioning

confidence: 99%

“…The investigation compares the phase‐based (Wadhwa et al., 2013) and inference‐based methods (Cai et al., 2022). This is because, among the state‐of‐the‐art methods based on video magnification (Cai & Zhang, 2022; Cai et al., 2020, 2022; Oh, et al., 2018), the inference‐based method can generate relatively the most accurate results. The detailed parameters utilized for all videos, such as the sampling frequency

{\rm{\;}}F{r_s}

, high‐pass frequency

{\rm{\;}}F{r_h}

, low‐pass frequency

F{r_l}

, modification factor α, resampling rates for non‐key frames

{R_n},

and key frames

{R_k}

, are given in Table 1.…”

Section: Case Studymentioning

confidence: 99%

“…The learning‐based method can produce more accurate motions, but much more specifically labeled datasets should be collected, and a lot of time is needed to train the neural networks (Chen & McDuff, 2020; Oh et al., 2018). Therefore, unsupervised methods such as optimization‐based methods are required to be developed (Cai & Zhang, 2022; Cai et al., 2020), which can reach an even better level of accuracy than the learning‐based method, for the adaptation of filters directly from optimization problems. However, their accuracy is restricted due to the lack of optimized phase shift rules.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Estimating small structural motions based on sparsity enforcement

Cai

Zhang

et al. 2022

Computer aided Civil Eng

Self Cite

View full text Add to dashboard Cite

This paper proposed a novel, efficient method to estimate challenging small structural motions from noisy video. To eliminate the phase limitation, ill-posed problem, and high computational burden, the structural motion function is resampled and recovered. Because video signals have tremendous redundancies in spatial, block, and time domains, the objective is achieved by enforcing three levels of sparsity constraints. In the first step, the ill-posed optimization is solved using the multi-hypothesis prediction embedded with the Tikhonov regularization, to resample and recover the video signal in the spatial domain. The second step is to characterize the measurement uncertainty in the block domain, in which two weight matrices are proposed into the regularization terms of the sparse coding. Finally, the temporal correlation of video frames is exploited by the reweighted residual sparsity. The superiority of the sparsity-enforcement method over existing methods was demonstrated through several case studies. Among the comparisons, the sparsity-enforcement method yielded video magnifications, structural motions, and modal information more accurately. Meanwhile, it has the lowest computational burden.

show abstract

Section: Sparsity‐enforcement Structural Motion Estimationmentioning

confidence: 99%

{\rm{\;}}F{r_s}

, high‐pass frequency

{\rm{\;}}F{r_h}

, low‐pass frequency

F{r_l}

, modification factor α, resampling rates for non‐key frames

{R_n},

and key frames

{R_k}

, are given in Table 1.…”

Section: Case Studymentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Estimating small structural motions based on sparsity enforcement

Cai

Zhang

et al. 2022

Computer aided Civil Eng

Self Cite

View full text Add to dashboard Cite

show abstract

“…More recently, a novel method using multifrequency absolute phase retrieval and fast cosine transform with extension to phase correction operation was proposed by [9]. Further advances based on Gaussian Mixture Model (GMM) learning algorithm permit it to improve the MM results with fewer artifacts and better anti-noise performance [10].…”

Section: Algorithms Usedmentioning

confidence: 99%

Motion Magnification Applications for the Protection of Italian Cultural Heritage Assets

Fioriti

Roselli

Cataldo

et al. 2022

Sensors

View full text Add to dashboard Cite

In recent years, the ENEA has introduced a novel methodology based on motion magnification (MM) into the Italian cultural heritage protection and monitoring field. It consists of a digital video signal processing technique able to amplify enormously the tiny movements recorded in conventional videos, while preserving the general topology of the acquired frames. Though the idea of such a methodology is not new, it has recently been provided with an efficient algorithm that makes possible a viable and low-cost magnification. Applications are extremely varied in almost every field of science and technology; however, we are interested in its application to the safeguarding of architectural heritage, a sector of the utmost importance for Italy. As ancient buildings can be extremely sensitive to even minimally invasive instrumentation, most common monitoring sensors can be replaced by contactless tools and methods, such as video-based techniques like MM. It offers many advantages: easy to use, contactless devices, virtual sensors, reusability of the videos, practicality, intuitive graphical results, quantitative analyses capability and low costs. These characteristics are well suited to the monitoring of large ancient monuments; on the other hand, historical sites have peculiarities of their own, requiring careful approaches, proper tools and trained personnel. Moreover, outdoor applications of MM present quite notable difficulties from a practical point of view, e.g., the dimensions of the studied objects, uncontrolled environmental conditions, spurious vibrations, lighting change/instability, etc. Here we give a general idea of the potential of MM and related issues, using some relevant in-the-field case studies in Italian heritage protection.

show abstract

“…Terefore, on the basis of considering the uncertainties, the process of updating the physical or simulation model of a structure emerges as a crucial issue and technical challenge within the feld of structural dynamics analysis in civil engineering [12][13][14]. Model updating serves as a structural health monitoring (SHM) approach, wherein modal parameters are identifed and updated [15,16] using vibration signals measured during structural tests.…”

Section: Introductionmentioning

confidence: 99%

A Bayesian Structural Modal Updating Method Based on Sparse Grid and Ensemble Kalman Filter

Lin,

Zhang,

Cai

et al. 2024

Structural Control and Health Monitoring

Self Cite

View full text Add to dashboard Cite

This study presents a sparse grid interpolation and ensemble Kalman filter (EnKF)-based Markov Chain Monte Carlo (MCMC) method (SG-EnMCMC). Initiating with the formulation of a recursive equation for the state space vector, derived from the structural dynamic equation, this study adopts a dimensionality reduction strategy. This approach involves a separation of physical parameters and the state space vector. The acquisition of physical parameters is accomplished through sampling, utilizing sample moments to substitute population moments, thereby mitigating the need for computationally high-dimensional covariance matrix calculations. To further streamline the recursive equation of the state space vector, a sparse grid method is employed for interpolation. This step simplifies the process while ensuring superior accuracy compared to the Extended Kalman Filter (EKF) and Unscented Kalman Filter (UKF). Subsequent to this, acceptance rates and the final parameter posterior distribution within the MCMC framework are derived. The efficiency of the proposed method is assessed through validation in two shaking table experiments.

show abstract

Gaussian mixture model based phase prior learning for video motion estimation

Cited by 13 publications

References 46 publications

Estimating small structural motions based on sparsity enforcement

Estimating small structural motions based on sparsity enforcement

Motion Magnification Applications for the Protection of Italian Cultural Heritage Assets

A Bayesian Structural Modal Updating Method Based on Sparse Grid and Ensemble Kalman Filter

Contact Info

Product

Resources

About