To image, or not to image: class-specific diffractive cameras with all-optical erasure of undesired objects

Abstract

Privacy protection is a growing concern in the digital era, with machine vision techniques widely used throughout public and private settings. Existing methods address this growing problem by, e.g., encrypting camera images or obscuring/blurring the imaged information through digital algorithms. Here, we demonstrate a camera design that performs class-specific imaging of target objects with instantaneous all-optical erasure of other classes of objects. This diffractive camera consists of transmissive surfaces structured using deep learning to perform selective imaging of target classes of objects positioned at its input field-of-view. After their fabrication, the thin diffractive layers collectively perform optical mode filtering to accurately form images of the objects that belong to a target data class or group of classes, while instantaneously erasing objects of the other data classes at the output field-of-view. Using the same framework, we also demonstrate the design of class-specific permutation and class-specific linear transformation cameras, where the objects of a target data class are pixel-wise permuted or linearly transformed following an arbitrarily selected transformation matrix for all-optical class-specific encryption, while the other classes of objects are irreversibly erased from the output image. The success of class-specific diffractive cameras was experimentally demonstrated using terahertz (THz) waves and 3D-printed diffractive layers that selectively imaged only one class of the MNIST handwritten digit dataset, all-optically erasing the other handwritten digits. This diffractive camera design can be scaled to different parts of the electromagnetic spectrum, including, e.g., the visible and infrared wavelengths, to provide transformative opportunities for privacy-preserving digital cameras and task-specific data-efficient imaging.

Massively parallel universal linear transformations using a wavelength-multiplexed diffractive optical network

“…For the implementation of the broadband diffractive designs in this paper, we fixed the mean value λm of this group of wavelengths {λ1,λ2,…,λNw}, i.e., λm=1Nw∑w=1Nwλw and assigned these wavelengths to be equally spaced between λ1=0.9125λm and λNw=1.0875λm. Unless otherwise specified, we chose λm to be 0.8 mm in our numerical simulations, as it aligns with the terahertz band that was experimentally used in several of our previous works 50 , 52 , 58 , 59 , 61 , 62 , 66 , 67 . Without loss of generality, the wavelengths used for the design of the broadband diffractive processors can also be selected at other parts of the electromagnetic spectrum, such as the visible band, for which the related simulation results and analyses can be found in Sec.…”

“…To mitigate some of these practical issues, various approaches, such as high-precision lithography and antireflection coatings can be utilized in the fabrication process of a diffractive network. As demonstrated in our previous work, 50 , 52 , 61 it is also possible to mitigate the performance degradation resulting from some of these experimental factors by incorporating them as random errors into the physical forward model used during the training process, which is referred to as “vaccination” of the diffractive network.…”

“…Unless otherwise specified, we chose λ m to be 0.8 mm in our numerical simulations, as it aligns with the terahertz band that was experimentally used in several of our previous works. 50,52,58,59,61,62,66,67 Without loss of generality, the wavelengths used for the design of the broadband diffractive processors can also be selected at other parts of the electromagnetic spectrum, such as the visible band, for which the related simulation results and analyses can be found in Sec. 3 to follow.…”

“…Diffractive neural networks have been used to perform various optical information processing tasks, including, object classification, 22 , 46 – 57 image reconstruction, 52 , 58 , 59 all-optical phase recovery and quantitative image phase imaging, 60 class-specific imaging, 61 super-resolution image displays, 62 and logical operations 63 – 65 Employing successive spatially engineered diffractive surfaces as the backbone for inverse design of deterministic optical elements also enabled numerous applications, such as spatially controlled wavelength demultiplexing, 66 pulse engineering, 67 and orbital angular momentum multiplexing/demultiplexing 68 …”

Class-specific diffractive cameras based on deep learning-designed surfaces

“…In a recent paper of eLight, the UCLA group introduced a new camera design by using diffractive computing 8 , which can image specific objects, such as MINST handwritten numbers, fashion items, while unwanted information was all-optically erased, as shown in Fig. 1.…”