Mobile Augmented Reality: User Interfaces, Frameworks, and Intelligence

Abstract

Mobile Augmented Reality (MAR) integrates computer-generated virtual objects with physical environments for mobile devices. MAR systems enable users to interact with MAR devices, such as smartphones and headworn wearables, and performs seamless transitions from the physical world to a mixed world with digital entities. These MAR systems support user experiences by using MAR devices to provide universal accessibility to digital contents. Over the past 20 years, a number of MAR systems have been developed, however, the studies and design of MAR frameworks have not yet been systematically reviewed from the perspective of user-centric design. This article presents the first effort of surveying existing MAR frameworks (count: 37) and further discusses the latest studies on MAR through a top-down approach: (1) MAR applications; (2) MAR visualisation techniques adaptive to user mobility and contexts; (3) systematic evaluation of MAR frameworks including supported platforms and corresponding features such as tracking, feature extraction plus sensing capabilities; and (4) underlying machine learning approaches supporting intelligent operations within MAR systems. Finally, we summarise the development of emerging research fields, current state-of-the-art, and discuss the important open challenges and possible theoretical and technical directions. This survey aims to benefit both researchers and MAR system developers alike. CCS Concepts: • Human-centered computing → Mixed / augmented reality; Graphical user interfaces; • Software and its engineering → Development frameworks and environments; • Computing methodologies → Machine learning approaches.

Benchmarking Built-In Tracking Systems for Indoor AR Applications on Popular Mobile Devices

“…This is also due to their versatility and ease of use. Among the best-known frameworks for rapid prototyping of AR solutions, ARKit ® and ARCore TM represent the most constantly evolving ones for iOS ® and Android TM devices, respectively [ 22 , 23 ]. They use a process called simultaneous localization and mapping (SLAM) to localize the device in real-world coordinates.…”