Exploring the Feature Space to Aid Learning in Design Space Exploration

Bang, Hyunseung; Shi, Yuan Ling Zi; Hoffman, Guy; Yoon, So‐Yeon; Selva, Daniel

doi:10.1007/978-3-030-05363-5_11

Cited by 2 publications

(1 citation statement)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Being based on ten binary questions, the test does not provide much power. In addition, we have observed significant variance in this and other experiments using similar tests [94]. Another limitation is that we are only testing Daphne with all its features versus a very limited version.…”

Section: F Discussionmentioning

confidence: 73%

Daphne: A Virtual Assistant for Designing Earth Observation Distributed Spacecraft Missions

Martin

Selva

2020

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

Self Cite

View full text Add to dashboard Cite

This article describes Daphne, a virtual assistant for designing Earth observation distributed spacecraft missions. It is, to the best of our knowledge, the first virtual assistant for such application. The article provides a thorough description of Daphne, including its question answering system and the main features we have implemented to help system engineers design distributed spacecraft missions. In addition, the article describes a study performed at NASA's Jet Propulsion Laboratory (JPL) to assess the usefulness of Daphne in this use case. The study was conducted with N = 9 subjects from JPL, who were asked to work on a mission design task with two versions of Daphne, one that was fully featured implementing the cognitive assistance functions, and one that only had the features one would find in a traditional design space exploration tool. After the task, they filled out a standard user experience survey, completed a test to assess how much they learned about the task, and were asked a number of questions in a semi-structured exit interview. Results of the study suggest that Daphne can help improve performance during system design tasks compared to traditional tools, while keeping the system usable. However, the study also raises some concerns with respect to a potential reduction in human learning due to the use of the cognitive assistant. The article ends with a list of suggestions for future development of virtual assistants for space mission design. Index Terms-Distributed spacecraft missions, earth observation, machine learning, mixed initiative, virtual assistant. I. INTRODUCTION M OTIVATED by the challenges of system architecture in general and architecting distributed satellite missions (DSM) in particular, and inspired by the success of commercial virtual assistants (VA), such as Siri, Google Assistant, Alexa, or Mycroft, we have developed Daphne, the first VA-to the best of our knowledge-to support the high-level design of DSM. This article describes how Daphne can be used to design DSM and includes a quantitative validation study performed at NASA JPL where the test subjects had expertise in mission design. The contribution of this article is two fold. First, we introduce Daphne as a complete, open-source package for DSM tradespace analysis that includes the standard VA functionality, such as Manuscript

show abstract

Section: F Discussionmentioning

confidence: 73%

Daphne: A Virtual Assistant for Designing Earth Observation Distributed Spacecraft Missions

Martin

Selva

2020

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

Self Cite

View full text Add to dashboard Cite

show abstract

Evaluating Designer Learning and Performance in Interactive Deep Generative Design

Chaudhari

Selva

2023

Journal of Mechanical Design

View full text Add to dashboard Cite

Deep generative models have shown significant promise in improving performance in design space exploration. But there is limited understanding of their interpretability, a necessity when model explanations are desired and problems are ill-defined. Interpretability involves learning design features behind design performance, called designer learning. This study explores human-machine collaboration's effects on designer learning and design performance. We conduct an experiment (N=42) designing mechanical metamaterials using a conditional variational auto-encoder. The independent variables are: (i) the level of automation of design synthesis, e.g., manual (where the user manually manipulates design variables), manual feature-based (where the user manipulates the weights of the features learned by the encoder), and semi-automated feature-based (where the agent generates a local design based on a start design and user-selected step size); and (ii) feature semanticity, e.g., meaningful versus abstract features. We assess feature-specific learning using Item Response Theory and design performance using utopia distance and hypervolume improvement. The results suggest design performance depends on the subjects' feature-specific knowledge, emphasizing the precursory role of learning. The semi-automated synthesis locally improves the utopia distance. Still, it does not result in higher global hypervolume improvement compared to manual design synthesis, and reduced designer learning compared to manual feature-based synthesis. The subjects learn semantic features better than abstract features only when design performance is sensitive to them. Potential cognitive constructs influencing learning in human-AI collaborative settings are discussed, such as cognitive load and recognition heuristics.

show abstract

Exploring the Feature Space to Aid Learning in Design Space Exploration

Cited by 2 publications

References 26 publications

Daphne: A Virtual Assistant for Designing Earth Observation Distributed Spacecraft Missions

Daphne: A Virtual Assistant for Designing Earth Observation Distributed Spacecraft Missions

Evaluating Designer Learning and Performance in Interactive Deep Generative Design

Contact Info

Product

Resources

About