Noncontact human-machine interaction based on hand-responsive infrared structural color

Abstract

Noncontact human-machine interaction provides a hygienic and intelligent approach for the communication between human and robots. Current noncontact human-machine interactions are generally limited by the interaction distance or conditions, such as in the dark. Here we explore the utilization of hand as an infrared light source for noncontact human-machine interaction. Metallic gratings are used as the human-machine interface to respond to infrared radiation from hand and the generated signals are visualized as different infrared structural colors. We demonstrate the applications of the infrared structural color-based human-machine interaction for user-interactive touchless display and real-time control of a robot vehicle. The interaction is flexible to the hand-interface distance ranging from a few centimeters to tens of centimeters and can be used in low lighting condition or in the dark. The findings in this work provide an alternative and complementary approach to traditional noncontact human-machine interactions, which may further broaden the potential applications of human-machine interaction.

Deep‐Learning‐Assisted Noncontact Gesture‐Recognition System for Touchless Human‐Machine Interfaces

“…Especially during the pandemic of coronavirus disease 2019 (COVID‐19), touchless HMIs can largely reduce the spread of virus. Current approaches for realizing touchless HMIs are mainly based on humidity, [ 9,10 ] infrared radiation, [ 11,12 ] capacitance, [ 13,14 ] magnetism, [ 15 ] electromagnetic, [ 16 ] and visual sensors. [ 17 ] Among them, the humidity, infrared radiation, and capacitance sensors can only detect limiting simple gesture types, such as approaching and sliding.…”

“…[ 17 ] Among them, the humidity, infrared radiation, and capacitance sensors can only detect limiting simple gesture types, such as approaching and sliding. [ 9,11,13 ] The magnetism sensor requires the wearing of magnet accessories, leading to a low interaction experience. [ 15 ] The electromagnetic sensor needs a complex sensing platform, [ 16 ] and the visual sensor cannot be used in low‐light environments like in the dark.…”

SrTiO3/CuNi‐Heterostructure‐Based Thermopile for Sensitive Human Radiation Detection and Noncontact Human–Machine Interaction

“…The active sensors, such as optical [ 4–6 ] and capacitive [ 7–9 ] proximity switches, comprise a continuous transmitter and a receiver, which typically require high power consumption. In contrast, the passive sensors which directly detect human‐related physical properties, such as spontaneous thermal radiation, [ 10,11 ] humidity, [ 12–15 ] and accumulated electric charges, [ 16,17 ] are known for their low‐power characteristic. For example, Tang et al.…”

“…The active sensors, such as optical [4][5][6] and capacitive [7][8][9] proximity switches, comprise a continuous transmitter and a receiver, which typically require high power consumption. In contrast, the passive sensors which directly detect human-related physical properties, such as spontaneous thermal radiation, [10,11] humidity, [12][13][14][15] and accumulated electric charges, [16,17] are known for their low-power characteristic. For example, Tang et al [16] utilized the electric charges gathered at the fingertip to induce the charge redistribution on the graphene surface, based on which a touchless screen sensor was constructed.…”

Full Geometric Control of Hidden Color Information in Diffraction Gratings under Angled White Light Illumination

“…Diffraction gratings are ubiquitous in optical systems with established applications in displays, − filters, − sensors, − and optical security devices. − Grating-based optical security devices are of particular interest, as they offer animated, three-dimensional, and rainbow effects , that are easy to verify and difficult to copy. Colors in gratings are produced by tuning geometric parameters − such as the height, periodicity, orientation, and angle of incidence.…”