Spatial and Temporal Enhancement of Depth Images Captured by a Time-of-Flight Depth Sensor

Overview of efficient high-quality state-of-the-art depth enhancement methods by thorough design space exploration

“…This causes depth errors and hence reduces the depth accuracy. Fu et al [14] and Kim et al [20] only partially solve this problem by filtering stationary image areas only. However, our test sequences have both camera and object motion and hence both approaches will not work.…”

“…Hence they are not suitable for implementation in real-time 3D-TV systems. In segmentation-based depth upsampling methods (Soh et al [38], Tallon et al [39] and Kim et al [20]), the color and bicubicly upsampled depth images are jointly segmented and regions where depth and color are inconsistent, are corrected. Generally these approaches require an additional post-processing step to remove the false depth transitions, introduced by segmentation.…”

“…Furthermore, Kim et al [20] estimate the motion of each pixel in the color image with a block matcher. Then if a pixel is stationary in both the previous and the current frame the depth of the previous frame is copied to that pixel, otherwise the upsampled depth of the current frame is kept.…”

“…20 As a result the throughputs of the Brox's based temporal post-processing methods are at least 35 to 9 times lower than their 3DRS-based counterparts. Figure 7 also shows that for kernel radii larger than 2, 3DJBU has a significantly lower throughput than JP, JPMC-3DRS and MCPF.…”

Real-time Motion-based Hand Gestures Recognition from Time-of-Flight Video

“…An example of the use of this technology can be found in [8] where it is used to improve people tracking in a smart room. TOF technology can also present some problems, such as optical noise existence, unmatched boundaries or temporal inconsistency [16]. The use of depth information results in an enrichment of the communication between user and machine by means of gestural interfaces.…”