ProGesAR: Mobile AR Prototyping for Proxemic and Gestural Interactions with Real-world IoT Enhanced Spaces

Fast-Forward Reality: Authoring Error-Free Context-Aware Policies with Real-Time Unit Tests in Extended Reality

“…CAPturAR [92] supports users to author activity-based context-aware applications when referring to their past activities that are visualized using AR avatars. By defining proxemics with physical surroundings using a mobile-device-based AR authoring interface, ProGesAR [101] and ProObjAR [102] enable users to include locations, object movements, and gestures into CAPs. Further, Ivy [29] immerses users into the digital twin of an indoor environment to define logic connections among smart objects via visual programming, while Haidon et al [36] enables a caregiver to author CAPs tailored to a resident's habits by selecting the remote resident's spatial-sensitive contexts that are mapped to the caregiver's local AR space using a semantic mapping approach.…”

“…Existing efforts have mainly targeted end-user challenges in authoring CAPs with specific context triggers and actions. Some prior research has supported users to manually test authored CAPs by either selecting the triggering contexts on a 2D GUI [26,74] or by demonstrating the human activities and proxemics in AR [92,101,102]. However, prior works do not investigate whether the user-authored CAPs perform according to user expectations in diversified run-time scenarios.…”

“…Furthermore, to fully support novice users in validating and refining their CAPs, how to present these test cases appropriately remains challenging as a majority portion of the contexts perceived by advanced AI modules are associated with the spatial information (e.g., locations [101], proximity [102], interactions [40,92], and IoT states [41]). However, the above-mentioned CAP validation tools do not address the need to enable local users to interpret the test cases precisely.…”

“…With trigger-action programming [84], end-users can author customized CAPs by specifying context factors as triggers (e.g., 'after sunset' and 'leaving home'), and smart functions as actions (e.g., 'turn on the lights' and 'turn off the A/C'). To address diversified user needs, researchers and commercial products have greatly expanded the range of context factors to include a variety of environmental identities (e.g., time, temperature, weather, smart objects [3,26,42,62,102]), as well as user's location [49,89,101], status [104], and activities [92,98]. While most existing tools focus on enabling end-users to author CAPs with great flexibility, a runtime issue emerges: it is challenging for users without programming skills to verify that the customized CAPs will behave as intended under varying contexts.…”

Make Interaction Situated: Designing User Acceptable Interaction for Situated Visualization in Public Environments

“…We adopt the term spatially-aware interaction as a scaleable method referring to[70,130]. In this paper, it specifically denotes the use of spatially-aware non-digital objects for the selected scenarios.…”