Smooth Operator

Abstract

Operators in closed-circuit television (CCTV) control rooms have to monitor large sets of video feeds coming from an ever increasing number of cameras. To assist these operators in their demanding day-to-day tasks, AI-driven support systems accompanied by user-centric interfaces are being developed. However, prototyping these support systems and testing them in operative control rooms can be a challenge. Therefore, in this paper, we present a virtual reality (VR) control room which can be used to investigate the effects of existing and future support systems on operators' performance and behaviour in a fully controlled environment. Important assets of this VR control room include the possibility to subject operators to different levels of cognitive load and to monitor their cognitive-affective states using not only subjective but also behavioural and physiological techniques.

Integrating Biofeedback and Artificial Intelligence into eXtended Reality Training Scenarios: A Systematic Literature Review

“…As such, details regarding integration of biosensing technologies with synthetic environment development tools (i.e., game engines) should be a focus of development approaches. The Unity game engine was used in several studies (including De Bruyne et al, 2021; Delvigne et al, 2022; Dwivedi et al, 2020; Marín-Morales et al, 2021; Porssut et al, 2022; Rodrigues et al, 2021; Stranick & Lopez, 2022; Tsai et al, 2021) and the Unreal Engine 4 was used in only one study from the 48 studies included (McDermott et al, 2022). Hardware and software integration is important for system development and Unity is currently dominating the “lock-in”, where platforms are made dependent on and interoperable with each other to assure market viability (Foxman, 2019).…”

“…A brief description of each of the ML/AI technologies considered is provided in Table 11. ( Abdurrahman et al, 2021), (Bhalla et al, 2021), (De Bruyne et al, 2021), (Delvigne et al, 2022), (Delvigne et al, 2020), (Georgiou & Demiris, 2017), (Jones et al, 2020), (Liebers et al, 2021), (Leiker et al, 2016), (Migovich et al, 2021), (Noghabaei et al, 2021), (Porssut et al, 2022), (Shi et al, 2020), (Tsai et al, 2021), (Wilson et al, 2021), (Zhang et al, 2017) From (Porssut et (Granato et al, 2020), (Ihmig et al, 2020), (Islam et al, 2019), (Kim et al, 2019), (Luong et al, 2020), (Martin et al, 2020), (Migovich et al, 2021), (Sakib et al, 2020), (Sayis et al, 2022), (Tabbaa et al, 2022), (Tsai et al, 2021), (Wilson et al, 2021), (Zhang et al, 2017) From (Martin et al, 2020) (continued) (Bauer et al, 2019), (Bin Suhaimi et al, 2020), (Delvigne et al, 2022), (Delvigne et al, 2020), (Jones et al, 2020), (Kaur et al, 2019), (Kim et al, 2020), (McDermott et al, 2022), …”

“…As such, details regarding integration of biosensing technologies with synthetic environment development tools (i.e., game engines) should be a focus of development approaches. The Unity game engine was used in several studies (including De Bruyne et al, 2021;Delvigne et al, 2022;Dwivedi et al, 2020;Marín-Morales et al, 2021; (Abdurrahman et al, 2021), (Abujelala et al, 2021), (Bhalla et al, 2021), (Bui et al, 2019), (De Bruyne et al, 2021), (Granato et al, 2020), (Kim et al, 2019), (Martin et al, 2020), (Marín-Morales et al, 2021), (Noghabaei et al, 2021), (Oh & Kim, 2021), (Recenti et al, 2021), (Rodrigues et al, 2021), (Stranick & Lopez, 2022), (Suhaimi et al, 2017), (Suhaimi et al, 2020), (Suhaimi, et al, 2022), (Tabbaa et al, 2022), (Zhang et al, 2017) + From "The Appen Artificial Intelligence Glossary", https://appen. Porssut et al, 2022;Rodrigues et al, 2021;Stranick & Lopez, 2022;Tsai et al, 2021) and the Unreal Engine 4 was used in only one study from the 48 studies included (McDermott et al, 2022).…”

I spy with my AI: The effects of AI-based visual cueing on human operators’ performance and cognitive load in CCTV control rooms

“…The second approach is to have the system draw rectangles around the person of interest in the camera stream that is pushed forward (Figure 1). In a previous study, a virtual CCTV control room was developed, where an operator's job was simulated and work pressure was manipulated (De Bruyne et al, 2021). Participants wore an HMD to interact with the virtual control room.…”

“…The results suggested that manipulating these two features (task difficulty and RSIs) altered cognitive load as measured by a subjective measure (i.e., NASA-TLX questionnaire) and physiological markers (e.g., eye-tracking). Therefore, the secondary task manipulation of (De Bruyne et al, 2021) offers the opportunity to investigate the effect of new support systems while operators are under different cognitive loads.…”