Object-based digital hologram segmentation and motion compensation

Abstract

Digital video holography faces two main problems: 1) computer-generation of holograms is computationally very costly, even more when dynamic content is considered; 2) the transmission of many high-resolution holograms requires large bandwidths. Motion compensation algorithms leverage temporal redundancies and can be used to address both issues by predicting future frames from preceding ones. Unfortunately, existing holographic motion compensation methods can only model uniform motions of entire 3D scenes. We address this limitation by proposing both a segmentation scheme for multi-object holograms based on Gabor masks and derive a Gabor mask-based multi-object motion compensation (GMMC) method for the compensation of independently moving objects within a single hologram. The utilized Gabor masks are defined in 4D space-frequency domain (also known as time-frequency domain or optical phase-space). GMMC can segment holograms containing an arbitrary number of mutually occluding objects by means of a coarse triangulation of the scene as side information. We demonstrate high segmentation quality (down to ≤ 0.01% normalized mean-squared error) with Gabor masks for scenes with spatial occlusions. The support of holographic motion compensation for arbitrary multi-object scenes can enable faster generation or improved video compression rates for dynamic digital holography.

Providing a Visual Understanding of Holography Through Phase Space Representations

“…For larger resolutions the error decreases fast. For multi-object holograms the ridge estimates get more complicated, but holographic scene segmentation [30] prior to the analysis can help. On a final note, also the lateral position along x of the scene center wrt.…”

“…Whilst this approach is based on numerically invertible PSRs applied to the full 2D holograms instead of visually interpretable PSRs applied to 1D cross-sections, it was motivated by a good knowledge of the phase space footprints of DHs. Hologram segmentation and subsequent multi-object motion compensation for dynamic holograms with the Gabor PSR, whose individual contributions are well localized in phase space, was demonstrated in [30]. Herein, we reprint in Figure 13 the phase space footprint of the individual objects after segmentation of the multi-object hologram used in Figure 8.…”

“…Several contributions by the authors have made use of phase space methods, too, e.g., for compression [27][28][29]; segmentation of holograms with multiple occluding objects [30]; faster computer generation [31]; development of more efficient propagation schemes [32]; comparison of angular and spatial multiplexed spatial-light modulator setups [33] and the study of their perceptual quality [34]; or to design orthoscopic display setups [35]. Despite this certainly non-exhaustive list of references it is evident how important phase space analysis and PSRs are in optics in general and in DH especially.…”

Compression strategies for digital holograms in biomedical and multimedia applications

“…In 84 , a motion compensation solution for holography based on the rotational transformation of wavefields 85 was used, where the residuals were further compressed using a STFT transform with an adaptive quantizer, and gains were demonstrated exceeding 20 dB over HEVC. Similarly, a spherical hologram based rotational compensation solution was proposed for an object moving with an arbitrary trajectory described with locally varying curvatures 86 . Unfortunately, all these methods are limited to a single moving object, and require application of object-based hologram segmentation techniques such as 87 to work for scenes with multiple objects.…”

Open access dataset of holographic videos for codec analysis and machine learning applications

“…However, one of the most important obstacles to the adoption of Holography for 3D visualization applications is the massive amount of information contained in digital holograms [2]. As a consequence, and since they have very different signal properties compared to conventional images and videos, specific compression techniques need to be designed for holographic data [3][4][5][6][7]. In this context, the Joint Photographic Experts Group (JPEG) designed a new standard for the compression of holograms, called JPEG Pleno Holography [8,9].…”