Spatio-temporal Upsampling for Free Viewpoint Video Point Clouds

A Self-regulating Spatio-Temporal Filter for Volumetric Video Point Clouds

“…The proposed system is an expansion to the work presented in [24] that is able to spatio-temporally upsample a point cloud sequence captured via widebaseline multi-view setups and further support the self-regulating noise filter metric using a new windowed approach to sampling. In summary, the following work proposes:…”

“…This is done via an edge-aware scene flow generated from the input RGB images. Expanding on [24], this is performed iteratively across a window of frames centered about the input frame i.e. we recursively project frames within the given window toward the center frame.…”

“…Table 1 provides a breakdown of the amount of pertinent geometry recovered via different optical flow techniques. [24] on the effect of STEA filter initialization on geometry recovered expressed as % increase in points. Tested on a synthetic ground-truth sequence.…”

“…Orientation is indicated by means of hue while vector magnitude is proportional to the saturation i.e. negligible motion is represented by white, high-speed motion is shown in highly saturated color [24]. defined by the Direct Linear Transformation algorithm [12].…”

“…However, this also has the effect of accumulating any inherent noise within this window. For this reason we extend the two-way Hausdorf filter in [24] with the addition of an energy density term E dens . This density term takes into account the average voxel density of the merged window of frames which is essentially the sum of the propagated clouds.…”