On the generation, structure, and semantics of grammar patterns in source code identifiers

Abstract

Identifiers make up a majority of the text in code. They are one of the most basic mediums through which developers describe the code they create and understand the code that others create. Therefore, understanding the patterns latent in identifier naming practices and how accurately we are able to automatically model these patterns is vital if researchers are to support developers and automated analysis approaches in comprehending and creating identifiers correctly and optimally. This paper investigates identifiers by studying sequences of partof-speech annotations, referred to as grammar patterns. This work advances our understanding of these patterns and our ability to model them by 1) establishing common naming patterns in different types of identifiers, such as class and attribute names; 2) analyzing how different patterns influence comprehension; and 3) studying the accuracy of state-of-the-art techniques for part-of-speech annotations, which are vital in automatically modeling identifier naming patterns, in order to establish their limits and paths toward improvement. To do this, we manually annotate a dataset of 1,335 identifiers from 20 open-source systems and use this dataset to study naming patterns, semantics, and tagger

An Ensemble Approach for Annotating Source Code Identifiers with Part-of-speech Tags

“…In addition, the ensemble has been made fully available (see Section IV), and is intended for long term support by the research team as we expand the training set and include identifiers from different contexts (e.g., test code). 2) Confirmation of observations we made in prior work [25] that indicate 1) the importance of the position of a word in an identifier, 2) the importance of the context of an identifier when annotating using part-of-speech, and 3) the complementarity of three part-of-speech taggers. 3) An expanded set of manually-annotated identifiers, based on the original set constructed in [25], that can be used to train and create other tagging approaches or for other natural language problems.…”

“…2) Confirmation of observations we made in prior work [25] that indicate 1) the importance of the position of a word in an identifier, 2) the importance of the context of an identifier when annotating using part-of-speech, and 3) the complementarity of three part-of-speech taggers. 3) An expanded set of manually-annotated identifiers, based on the original set constructed in [25], that can be used to train and create other tagging approaches or for other natural language problems. As with the implementation, this has been made fully available to the research community (see Section IV).…”

“…Part-of-speech tagging is one of the most popular methods for measuring the natural language semantics of identifier names and has been used in numerous other research [19], [17], [20], [21], [22], [23], [14], [13], [24]. Unfortunately, part-of-speech taggers for identifiers are still inaccurate [25], [18], making it difficult to trust their output.…”

“…1) An implementation of the most accurate (to-date) part-ofspeech tagger for source code identifiers, built using data that was curated via significant manual-annotation effort made by the authors in prior work [25]. This approach is trained to support more types of annotations (i.e., POS tags) specifically oriented for source code than any other approach currently available.…”

“…As with the implementation, this has been made fully available to the research community (see Section IV). 4) A thorough evaluation of the ensemble approach at both the identifier-and word-level, including a discussion of the features, which were empirically derived by the authors in prior work [25], that most positively influence the tagger's performance. 5) A discussion that provides a clear path for future work on part-of-speech tagger accuracy and effectiveness.…”