Actuation Confirmation and Negation via Facial-Identity and -Expression Recognition

Abstract

This paper presents the implementation of a facial-identity and-expression recognition mechanism that confirms or negates physical and/or computational actuations in an intelligent built-environment. Said mechanism is built via Google Brain's TensorFlow (as regards facial identity recognition) and Google Cloud Platform's Cloud Vision API (as regards facial gesture recognition); and it is integrated into the ongoing development of an intelligent built-environment framework, viz., Design-to-Robotic-Production &-Operation (D2RP&O), conceived at Delft University of Technology (TUD). The present work builds on the inherited technological ecosystem and technical functionality of the Design-to-Robotic-Operation (D2RO) component of said framework; and its implementation is validated via two scenarios (physical and computational). In the first scenario-and building on an inherited adaptive mechanism-if building-skin components perceive a rise in interior temperature levels, natural ventilation is promoted by increasing degrees of aperture. This measure is presently confirmed or negated by a corresponding facial expression on the part of the user in response to said reaction, which serves as an intuitive override / feedback mechanism to the intelligent building-skin mechanism's decision-making process. In the second scenario-and building on another inherited mechanism-if an accidental fall is detected and the user remains consciously or unconsciously collapsed, a series of automated emergency notifications (e.g., SMS, email, etc.) are sent to family and/or caretakers by particular mechanisms in the intelligent built-environment. The precision of this measure and its execution are presently confirmed by (a) identity detection of the victim, and (b) recognition of a reflexive facial gesture of pain and/or displeasure. The work presented in this paper promotes a considered relationship between the architecture of the builtenvironment and the Information and Communication Technologies (ICTs) embedded and/or deployed.

Design and Development of a Novel Robotic Gripper for Automated Scaffolding Assembly

“…Furthermore, said mechanism is one of several cloud-based mechanisms subsumable by the system that enhance HCI. For example, the authors have previously developed Machine Learning (ML)based Human Activity Recognition [21] and Object and Facial-Identity and -Expression Recognition mechanisms [22], two mechanisms that may be implemented into the present system in order to increase context-awareness and interaction pertinence.…”

Development of An Adaptive Staircase System Actuated by Facial-, Object-, and Voice-Recognition

“…The mechanical consists of the physical parts that instantiate the reconfiguration modes particular to each user-type. Each of the computational mechanisms inherits and/or builds upon previous developments via Application Programming Interfaces (APIs) by the authors (with respect to facial- [7], object- [8], and voice-recognition [9]). That is to say:…”

The Intelligent Built-Environment as Cyber-Physical System

“…Said nodes continuously send and receive information with microcontrollers (MCUs) embedded in the interior space that serve a variety of other service features inherited from previous work. Some of these MCUs, which are strategically installed in specific locations within the interior space, are equipped with a Raspberry Pi Camera v2 capable of engaging in computer vision functions such as facial-recognition [175] as well as gesture-recognition. A user in the interior of a built-environment covered by the service range of a given set of nodes (again, depending on where the nodes are strategically installed considering the building's geolocation) engages the light-tracking and -redirecting system by effecting an initializing hand-gesture (viz., Figure 5.19, Gesture 1).…”