Bespoke: Interactively Synthesizing Custom GUIs from Command-Line Applications By Demonstration

Abstract

Programmers, researchers, system administrators, and data scientists often build complex workflows based on commandline applications. To give these power users the well-known benefits of GUIs, we created Bespoke, a system that synthesizes custom GUIs by observing user demonstrations of command-line apps. Bespoke unifies the two main forms of desktop human-computer interaction (command-line and GUI) via a hybrid approach that combines the flexibility and composability of the command line with the usability and discoverability of GUIs. To assess the versatility of Bespoke, we ran an open-ended study where participants used it to create their own GUIs in domains that personally motivated them. They made a diverse set of GUIs for use cases such as cloud computing management, machine learning prototyping, lecture video transcription, integrated circuit design, remote code deployment, and gaming server management. Participants reported that the benefit of these bespoke GUIs was that they exposed only the most relevant subset of options required for their specific needs. In contrast, vendor-made GUIs usually include far more panes, menus, and settings since they must accommodate a wider range of use cases.

FindCmd: A personalised command retrieval tool

“…Bespoke [7] synthesises custom GUIs by observing user demonstrations of command-line apps; it combines the flexibility and composability of the command line with the usability and discoverability of GUIs.…”

“…All the discussed techniques have their own ways of improving the convenience of command use. However, works [4][5][6][7] cannot help the users to find the needed command, the local-based retrieval tools are implemented based on the string searching and do not perform well, whereas the web-based retrieval tools ignore the factors of local machine and user skills and they rely on the network.…”

An empirical investigation of command-line customization

“…We see our large-scale analysis on command-line user customizations manifested in alias definitions as a unique window of opportunity to study how the standard environment of the command line could be productively extended, modified, and improved. Our work goes hand in hand with existing efforts to innovate on the experience of command lines that employ techniques from research in systems (Raghavan et al 2020;Handa et al 2021), software engineering and programming languages (Vasilakis et al 2021;Vasilakis et al 2020;D'Antoni et al 2017), human-computer interaction (Vaithilingam and Guo 2019;Gandhi and Gandhi 2020), and artificial intelligence (Agarwal et al 2020;Lin et al 2018;Hou et al 2021). Particularly, our extensive qualitative and quantitative analysis, in conjunction with our dataset, form the basis for identifying opportunities for improving command-line experience in the following directions: by characterizing customization practices, we gain a categorical understanding underlying the needs and wants of command-line users; based on our analysis, we identify opportunities for innovation and formulate them as implications, accompanied with concrete scenarios and examples; further, our comprehensive dataset enables the foundation of learning approaches, as part of learning-based program synthesis (Bruch et al 2009;Raychev et al 2014), automated repair (Monperrus 2018), and recommendation systems (Mens and Lozano 2014); finally, we also see our results and datasets as a basis for usability research that can impact the design of tools and the future of the shell in general.…”

“…Khosmood et al (2014) use the same corpus and two additional, more recent, corpora to learn a model that can identify user profiles based on their command-line behavior. Bespoke (Vaithilingam and Guo 2019) is a system that synthesizes specialized graphical user interfaces (GUIs) based on command usage. Our work can be viewed as an input to this system that passes common shell workflows in aliases to be generated as GUIs.…”

An Empirical Investigation of Command-Line Customization

“…[18] use the same corpus and two additional, more recent, corpora to learn a model that can identify user profiles based on their command-line behavior. Bespoke [28] is a system that synthesizes specialized graphical user interfaces (GUIs) based on command usage. Our work can be viewed as an input to this system that passes common shell workflows in aliases to be generated as GUIs.…”