Automated Concept Location Using Independent Component Analysis

As development on a software project progresses, developers shift their focus between different topics and tasks many times. Managers and newcomer developers often seek ways of understanding what tasks have recently been worked on and how much effort has gone into each; for example, a manager might wonder what unexpected tasks occupied their team's attention during a period when they were supposed to have been implementing new features. Tools such as Latent Dirichlet Allocation (LDA) and Latent Semantic Indexing (LSI) can be used to extract a set of independent topics from a corpus of commit-log comments. Previous work in the area has created a single set of topics by analyzing comments from the entire lifetime of the project. In this paper, we propose windowing the topic analysis to give a more nuanced view of the system's evolution. By using a defined time-window of, for example, one month, we can track which topics come and go over time, and which ones recur. We propose visualizations of this model that allows us to explore the evolving stream of topics of development occurring over time. We demonstrate that windowed topic analysis offers advantages over topic analysis applied to a project's lifetime because many topics are quite local.

show abstract

“…Grant et al [5] and have used an alternative technique, called Independent Component Analysis [3] to separate topic signals from software code.…”

Section: Lda Lsi and Semantic Clusteringmentioning

confidence: 99%

What's hot and what's not: Windowed developer topic analysis

Hindle

Godfrey

Holt

2009

2009 IEEE International Conference on Software Maintenance

View full text Add to dashboard Cite

show abstract

“…Our previous work in this area [6] has shown that individual concepts found in software can be isolated using Figure 1. ICA Matrices this technique.…”

Section: Introductionmentioning

confidence: 99%

Vector space analysis of software clones

Grant

Cordy

2009

2009 IEEE 17th International Conference on Program Comprehension

Self Cite

View full text Add to dashboard Cite

In this paper, we introduce a technique for applying Independent Component Analysis to vector space representations of software code fragments such as methods or blocks. The distance between these points can be determined, and used as a measure of the similarity between the original source code fragments they represent. It can be reasoned that if the initial matrix representation contains enough information about the syntactic structure of the source code, the vector space representation will be sufficient to predict the similarity of fragments to one another, and can provide the likelihood that the code is a clone.

show abstract

“…A significant improvement over regular expression matching is brought by information retrieval-based [Cleary et al 2009;Gay et al 2009;Liu et al 2007;Marcus et al 2004;Poshyvanyk et al 2006b] and natural language processing Hill et al 2009;Shepherd 2007;Shepherd et al 2007] approaches, which allow more general textual queries and provide ranking of the results to these queries. Recent approaches also utilized independent component analysis [Grant et al 2008], and Latent Dirichlet Allocation [Lukins et al 2008] to support concept location. One common limitation of these approaches is redundancy in the search results that could be reduced by inferring additional structure among the search results and grouping them based on common topics.…”

Section: Related Workmentioning

confidence: 99%

Concept location using formal concept analysis and information retrieval

Poshyvanyk

Gethers

Marcus

2012

ACM Trans. Softw. Eng. Methodol.

View full text Add to dashboard Cite

________________________________________________________________________The paper addresses the problem of concept location in source code by proposing an approach that combines Formal Concept Analysis and Information Retrieval. In the proposed approach, Latent Semantic Indexing, an advanced Information Retrieval approach, is used to map textual descriptions of software features or bug reports to relevant parts of the source code, presented as a ranked list of source code elements. Given the ranked list, the approach selects the most relevant attributes from the best ranked documents, clusters the results, and presents them as a concept lattice, generated using Formal Concept Analysis.The approach is evaluated through a large case study on concept location in the source code on six opensource systems, using several hundred features and bugs. The empirical study focuses on the analysis of various configurations of the generated concept lattices and the results indicate that our approach is effective in organizing different concepts and their relationships present in the subset of the search results. In consequence, the proposed concept location method has been shown to outperform a standalone Information Retrieval based concept location technique by reducing the number of irrelevant search results across all the systems and lattice configurations evaluated, potentially reducing the programmers' effort during software maintenance tasks involving concept location.

show abstract

Automated Concept Location Using Independent Component Analysis

Cited by 40 publications

References 8 publications

What's hot and what's not: Windowed developer topic analysis

What's hot and what's not: Windowed developer topic analysis

Vector space analysis of software clones

Concept location using formal concept analysis and information retrieval

Contact Info

Product

Resources

About