ImageSense: An Intelligent Collaborative Ideation Tool to Support Diverse Human-Computer Partnerships

Abstract

Professional designers create mood boards to explore, visualize, and communicate hard-to-express ideas. We present ImageSense, an intelligent, collaborative ideation tool that combines individual and shared work spaces, as well as collaboration with multiple forms of intelligent agents. In the collection phase, ImageSense offers fluid transitions between serendipitous discovery of curated images via ImageCascade, combined text-and image-based Semantic search, and intelligent AI suggestions for finding new images. For later composition and reflection, ImageSense provides semantic labels, generated color palettes, and multiple tag clouds to help communicate the intent of the mood board. A study of nine professional designers revealed nuances in designers' preferences for designer-led, system-led, and mixed-initiative approaches that evolve throughout the design process. We discuss the challenges in creating effective human-computer partnerships for creative activities, and suggest directions for future research. CCS Concepts: • Human-centered computing → Interactive systems and tools; Computer supported cooperative work; • Applied computing → Arts and humanities.

Generative Theories of Interaction

“…Our studies show that mood designers' needs vary: sometimes they can articulate what they are looking for; sometimes they are stuck and need inspiration; and sometimes they want to successively refine their ideas, alternating between offering and receiving suggestions. We designed ImageSense [93] as a human-computer partnership that offers all three types of intelligent assistance across the collection, composition, and reflection phases. The key challenge is how to let designers control the level of user-and system-based agency, such that the system's actions benefit designers, without interrupting them.…”

Complex Problem Solving through Human-AI Collaboration: Literature Review on Research Contexts

“…Design. Five studies are related to design teams supported by AI systems, e.g., focusing on the creation of mood boards ("visual collages composed of images, text, and objects, that express concepts, ideas and emotions" [24]) or on engineers designing bridges [25] or drones [26]. While the humans are in the lead, the AI system provides suggestion for further ideas or improvements.…”

“…While the humans are in the lead, the AI system provides suggestion for further ideas or improvements. AI systems were chosen to support in those research contexts due to their high task-specific performance expected to be effective when combined with human creativity and agility [25] and their ability to support the entire creative process including inspiration and reflection [24].…”

“…mission success [38] PB: Protecting or sacrificing teammate [22,23] AP: Player identity (humanness) [31], perceived helpfulness, efficiency [37], trust, understandability [38] I-TC: AI vs. human [31,36] I-TC: AI vs. presumed-human [22,23] I-E: Score visible [23] M-TP: Communication: different modalities [36]/ explanation types [37,38]/ skill, reliability [38] Dependability: Visibility of score on treatment of AI teammate [23] Humans act differently in presumed social context [22] Online players value co-presence [36] Positive effect: Staying close to other player on correct identification (AI or human) [31] Positive effect: explanation on non-experts' performance [37]/on performance if facilitating decision-making [38] Design Mood board-, bridge-, drone-, accessory-, game level-design [24-26, 39, 40] PO: Strength mass ratio [25], quality, novelty [40], range, velocity, payload, cost [26] PB: Design effort, search strategy, mental workload [26], solution exploration [24] AP: Usefulness, inspirational [24], fun,…”

SemanticCollage

“…Semantic-Collage could also enrich the collaborative aspects of creative ideation and sense making. A first step is ImageSense [26], a collaborative mood board design tool where designers and machine learning algorithms share agency in the exploration, sharing, and communication of ideas.…”