Generation Probabilities Are Not Enough: Exploring the Effectiveness of Uncertainty Highlighting in AI-Powered Code Completions

Abstract

Large-scale generative models enabled the development of AI-powered code completion tools to assist programmers in writing code. However, much like other AI-powered tools, AI-powered code completions are not always accurate, potentially introducing bugs or even security vulnerabilities into code if not properly detected and corrected by a human programmer. One technique that has been proposed and implemented to help programmers identify potential errors is to highlight uncertain tokens. However, there have been no empirical studies exploring the effectiveness of this technique-nor investigating the different and not-yet-agreed-upon notions of uncertainty in the context of generative models. We explore the question of whether conveying information about uncertainty enables programmers to more quickly and accurately produce code when collaborating with an AI-powered code completion tool, and if so, what measure of uncertainty best fits programmers' needs. Through a mixed-methods study with 30 programmers, we compare three conditions: providing the AI system's code completion alone, highlighting tokens with the lowest likelihood of being generated by the underlying generative model, and highlighting tokens with the highest predicted likelihood of being edited by a programmer. We find that highlighting tokens with the highest predicted likelihood of being edited leads to faster task completion and more targeted edits, and is subjectively preferred by study participants. In contrast, highlighting tokens according to their probability of being generated does not provide any benefit over the baseline with no highlighting. We further explore the design space of how to convey uncertainty in AI-powered code completion tools, and find that programmers prefer highlights that are granular, informative, interpretable, and not overwhelming. This work contributes to building an understanding of what uncertainty means for generative models and how to convey it effectively.

AI Transparency in the Age of LLMs: A Human-Centered Research Roadmap

“…What is a useful notion of uncertainty for LLMs? While LLMs have a notion of uncertainty baked into them -the likelihood that the model would generate a specific token given its preceding or surrounding context (Bengio et al, 2003), what we have referred to in past work as the generation probability (Vasconcelos et al, 2023)-whether this notion would be useful to different stakeholders is questionable. In particular, this notion may not line up with people's intuition about what it means for the model to be uncertain.…”

“…Carefully selecting a notion of uncertainty to convey to stakeholders matters because the particular notion used impacts their behavior and trust. In our recent work with collaborators (Vasconcelos et al, 2023) provided in (e.g., its precision and modality), and what the effect is (e.g., on trust or behaviors), as well as taking into consideration the characteristics of the receiver (Van Der Bles et al, 2019). For example, in our study on uncertainty in the context of code completion tools (Vasconcelos et al, 2023), by soliciting participants' feedback on different uncertainty communication design choices, we found that programmers prefer uncertainty about granular or meaningful blocks to guide them to make token-level changes and prefer less precise communication (as opposed to exact quantification) for easy processing-both ultimately supporting their goal of producing correct code efficiently.…”

Designerly Understanding: Information Needs for Model Transparency to Support Design Ideation for AI-Powered User Experience

“…What is a useful notion of uncertainty for LLMs? While LLMs have a notion of uncertainty baked in them-the likelihood that the model would generate a specific token given its preceding or surrounding context [17], what we have referred to in past work as the generation probability [178]whether this notion would be useful to different stakeholders is questionable. In particular, this notion may not line up with people's intuition about what it means for the model to be uncertain.…”

“…Carefully selecting a notion of uncertainty to convey to stakeholders matters because the particular notion used impacts their behavior and trust. In our recent work with collaborators [178], we explored the effectiveness of displaying two alternative notions of uncertainty to programmers interacting with an LLM-powered code completion tool. In a mixed-methods study with 30 programmers, we compared three conditions: providing a code completion alone, highlighting those tokens with the lowest likelihood of being generated by the underlying LLM (i.e., lowest generation probability), and highlighting tokens with the highest predicted likelihood of being edited by a programmer according to a separate "edit model" trained on logged data from past programmer interactions.…”

“…The social science literature suggests that choosing an effective form of uncertainty communication requires articulating what the uncertainty is regarding (e.g., uncertainty about an individual token or about a full output, and which source of uncertainty), what form it is provided in (e.g., its precision and modality), and what the effect is (e.g., on trust or behaviors), as well as taking into consideration of the characteristics of the receiver [177]. For example, in our study on uncertainty in the context of code completion tools [178], by soliciting participants' feedback on different uncertainty communication design choices, we found that programmers prefer uncertainty about granular or meaningful blocks to guide them to make token-level changes and prefer less precise communication (as opposed to exact quantification) for easy processing-both ultimately supporting their goal of producing correct code efficiently.…”