Going Beyond Second Screens: Applications for the Multi-display Intelligent Living Room

Abstract

This work aims to investigate how the amenities offered by Intelligent Environments can be used to shape new types of useful, exciting and fulfilling experiences while watching sports or movies. Towards this direction, two ambient media players were developed aspiring to offer live access to secondary information via the available displays of an Intelligent Living Room, and to appropriately exploit the technological equipment so as to support natural interaction. Expert-based evaluation experiments revealed some factors that can influence the overall experience significantly, without hindering the viewers' immersion to the main media.

The User Experience of Journeys in the Realm of Augmented Reality Television

“…For example, the Mixed Reality TV Mozaik [4] runs on HoloLens as well as on a smartphone, TV+HoloLens [80] displays a sign language interpreter next to the TV, and Saeghe et al [68] used HoloLens to synchronize holograms with televised content. Other implementations of ARTV have employed video projections, either on the wall behind the TV [35,42,72,73] or in the entire room [34]. Examples include Point & Click [72], a spatial AR system enabling users to position virtual TV screens on the wall behind the conventional TV set, IllumiRoom [35] and ExtVision [38] that expand the content shown on the TV with video projections on the wall behind it, Smart Wallpaper [7] for electronic wallpaper that can also show TV content, and the Around TV [73] spatial AR system with a graphical user interface composed of virtual TV screens, menus, and control widgets video projected on the wall behind the TV set.…”

“…Examples include Point & Click [72], a spatial AR system enabling users to position virtual TV screens on the wall behind the conventional TV set, IllumiRoom [35] and ExtVision [38] that expand the content shown on the TV with video projections on the wall behind it, Smart Wallpaper [7] for electronic wallpaper that can also show TV content, and the Around TV [73] spatial AR system with a graphical user interface composed of virtual TV screens, menus, and control widgets video projected on the wall behind the TV set. Some systems have combined multiple displays, such as screens, walls, and interactive tables [42,76], and everyday objects as new media [31] to enrich the TV watching experience. For more information, we refer to Saeghe et al's [67] review of ARTV themes.…”

“…Also, the expected level of the augmentation, from a simple visual notifcation (e.g., a popup message rendered in smartglasses about breaking news on channel "6") to stylized content to realistic looking scenery and characters to photometric lighting and refections integrating the physical space (e.g., photorealistic content that adopts the lighting sources from the living room), is an important aspect when designing appropriate vehicles to deliver augmented TV content. This also includes choices regarding AR/VR display technology, from simple on-TV augmentations [53,74] to second screens and mobile devices [4,9,18] to smartglasses and HMDs [4,80,81] to video projections in the living room [35,42,73]. Important future work is to outline functional specifcations for various regions of the ARTV continuum together with requirements for the content as well as the AR/VR rendering technology to deliver that content.…”

“…For example, consider a user watching a soccer game broadcast on the TV screen from their living room. The user activates the display of additional content in AR, such as information about the two teams or the silhouette of a live sports commentator shown next to the TV screen rendered with a video projection [42,73] or through a HMD [68,80]. Replays of key moments from the game can be summoned in the form of immersive holograms in the living room, unfolding the action of the game from diferent viewpoints [14], e.g., of the striker, the goalkeeper, or the referee.…”