Key frame extraction algorithms consider the problem of selecting a subset of the most informative frames from a video to summarize its content. Several applications, such as video summarization, search, indexing, and prints from video, can benefit from extracted key frames of the video under consideration. Most approaches in this class of algorithms work directly with the input video data set, without considering the underlying low-rank structure of the data set. Other algorithms exploit the low-rank component only, ignoring the other key information in the video. In this paper, a novel key frame extraction framework based on robust principal component analysis (RPCA) is proposed. Furthermore, we target the challenging application of extracting key frames from unstructured consumer videos. The proposed framework is motivated by the observation that the RPCA decomposes an input data into: 1) a low-rank component that reveals the systematic information across the elements of the data set and 2) a set of sparse components each of which containing distinct information about each element in the same data set. The two information types are combined into a single l1-norm-based non-convex optimization problem to extract the desired number of key frames. Moreover, we develop a novel iterative algorithm to solve this optimization problem. The proposed RPCA-based framework does not require shot(s) detection, segmentation, or semantic understanding of the underlying video. Finally, experiments are performed on a variety of consumer and other types of videos. A comparison of the results obtained by our method with the ground truth and with related state-of-the-art algorithms clearly illustrates the viability of the proposed RPCA-based framework.

Automatic video summarization is indispensable for fast browsing and efficient management of large video libraries. In this paper, we introduce an image feature that we refer to as heterogeneity image patch (HIP) index. The proposed HIP index provides a new entropy-based measure of the heterogeneity of patches within any picture. By evaluating this index for every frame in a video sequence, we generate a HIP curve for that sequence. We exploit the HIP curve in solving two categories of video summarization applications: key frame extraction and dynamic video skimming. Under the key frame extraction frame-work, a set of candidate key frames is selected from abundant video frames based on the HIP curve. Then, a proposed patch-based image dissimilarity measure is used to create affinity matrix of these candidates. Finally, a set of key frames is extracted from the affinity matrix using a min–max based algorithm. Under video skimming, we propose a method to measure the distance between a video and its skimmed representation. The video skimming problem is then mapped into an optimization framework and solved by minimizing a HIP-based distance for a set of extracted excerpts. The HIP framework is pixel-based and does not require semantic information or complex camera motion estimation. Our simulation results are based on experiments performed on consumer videos and are compared with state-of-the-art methods. It is shown that the HIP approach outperforms other leading methods, while maintaining low complexity.

No abstract

No abstract

No abstract

A novel manifold method of reconstructing dynamically evolving spatial fields is presented for assimilating data from sensor networks in integrated land surfacesubsurface, oceanic / lake models. The method was developed based on the assumption that data can be mapped onto an underlying differential manifold. In this study, the proposed method was used to reconstruct meteorological forcing over Lake Michigan, the bathymetry of an inland lake (Gull Lake), and precipitation over the Grand River watershed in Michigan. In the first case study, hourly interpolated meteorological forcing data were used to run a three-dimensional hydrodynamic model of Lake Michigan to quantify the improvement that results from the use of the new interpolation method. In the second example, the bathymetry of Gull Lake was reconstructed from measured scatter point data using the manifold technique. A hydrodynamic model of Gull Lake was developed and further improved using improved bathymetry. In the last case study, daily participation data over a six-year period were interpolated over the Grand River watershed and used as input to an integrated, distributed hydrologic model. All three examples illustrate the superior performance of the manifold method over standard methods in terms of accuracy and computational efficiency. Our results also indicate that evaluating the relative performance of interpolation methods using the cross-validation method can lead to misleading conclusions about the relative performance of the methods.In situ observations generally have sparse and inhomogeneous distribution in space and time, 52 and it is often infeasible to accurately reconstruct the true field from the data. However, more 53 information about the structure of the field and its evolution, allows for better approximations 54 (Barth et al., 2008). Various deterministic [e.g., nearest neighbor, natural neighbor, inverse 55 distance weighting (IDW), spline, polynomial] and geostatistical (e.g. kriging) interpolation 56 methods have been developed to generate spatial fields. There have been numerous efforts to 57 compare different spatial interpolation methods in order to identify the best method for a given 58 model application. Many researches have used cross-validation for assessing the performance of 59 the interpolation methods. In this method, a subset of the original dataset is withheld to be used 60 later for validating the interpolated field constructed from the rest of the observational data. 61Mean error (ME), root mean square error (RMSE) and the coefficient of determination (R 2 ) are 62 commonly used to evaluate the performance of each interpolation method (Suparta and Rahman, 63 2016). However, every problem has a unique method of interpolation that works best for a given 64 distribution of observations and the intended use of the interpolated data. Density of a sensor 65 network, spatial variability of the variable of interest and its distribution, and observational 66 errors, all influence the accuracy of the interpolated field (MacEa...

No abstract

No abstract