Multimodal capture of consumer intent in retail

Abstract

In this paper we present a prototype for capturing retail related consumer intent using multiple devices and in multimodal input formats such as text, audio, and still images. The prototype was used in a longitudinal user study to analyze the process that consumers go through in order to make purchasing decisions. Based on these findings, we recommend desirable features for information management systems specifically designed for the retail environment.

Discovering user profiles

“…This was obtained using user profile aggregation tools we've built to collect calendar/event information (from Google Calendar using the GData services), from FOAF social network information, and from explicit user intent from our user intent capture framework [5]. Profile manager functionality is exposed via REST Web Services.…”

Snap and match

“…The three most popular mechanisms of choosing a foundation shade were -visual selection (33%), help at a cosmetic counter at a high-end store (30%), and testers at a high-end store (25%) [6].…”

Color match

“…Thus the goals of the system were to provide: a) expert makeup advice; b) the ability to choose foundation without relying on inaccurate printed brochures or visual inspection of shades in a product bottle (visual inspection tends to be problematic since the shade of the foundation differs from what is seen in a bottle to what it appears like after it dries down on application, also the product is typically packed in layers of packaging material); c) the ability to get advice anytime and anywhere since shopping is typically a collaborative and social activity [6].…”