Declarative AR and image processing on the web with Xflow

Abstract

Figure 1: Left image: an AR application developed with XML3D and Xflow. The displayed teapot jumps from one visible marker to the other. Center image: another AR application. This time, animated characters are displayed on top of the markers (character models by Valve Software). Right image: several image processing operators implemented with Xflow. (from the top left: AbstractRecently, modern Web browser became capable of supporting powerful, interactive 3D graphics both via the low-level, imperative API of WebGL as well as via a high-level, declarative approach like XML3D. The obvious next step (particularly with respect to mobile platforms) is to combine video from the real world with matched virtual content -Augmented or Mixed Reality (AR/MR). However, AR requires extensive image or video processing, feature detection and tracking, and applying the results to 3D renderingall of which is hard to implement in a Web context.In this paper we present a novel approach that encapsulates lowlevel image-processing and AR operations into re-usable high-level XML3D/Xflow components that are part of the HTML-5 DOM. A Web developer can then easily and flexibly arrange these components into (possibly complex) processing flow-graphs without having to worry about the internal computations and the structure of these modules. Our extended Xflow implementation automatically optimizes, schedules, and synchronizes the processing of the flow graph(s) in the context of real-time 3D rendering. Finally, we provide an integration model that greatly simplifies building AR applications for the browser.We demonstrate this with several simple AR and image processing applications using a polyfill implementation working in all modern browsers and evaluate the performance. Finally, we show how the declarative framework can be optimized with respect to performance and usability using parallelization with Web Workers and RiverTrail.

A2W: Context-Aware Recommendation System for Mobile Augmented Reality Web Browser

“…AR Web Browsers: A User Perspective. AR applications can be categorised into two types: (1) Leveraging the web technologies for the implementation of platform-specific AR browsers [34,43], and (2) solely web browsers [20,36]. Although the existing works facilitate the AR content generation and rendering of static animated contents [32], the optimized design of AR layout, including visual content placement and user experience is neglected [24].…”