Using Decision Tree Classifiers in Source Code Analysis to Recognize Algorithms: An Experiment with Sorting Algorithms

Abstract

We discuss algorithm recognition (AR) and present a method for recognizing algorithms automatically from Java source code. The method consists of two phases. In the first phase, the recognizable algorithms are converted into the vectors of characteristics, which are computed based on static analysis of program code, including various statistics of language constructs and analysis of Roles of Variables in the target program. In the second phase, the algorithms are classified based on these vectors using the C4.5 decision tree classifier. We demonstrate the performance of the method by applying it to sorting algorithms. Using leave-one-out cross-validation technique, we have conducted an experimental evaluation of the classification performance showing that the average classification accuracy is 98.1% (the data set consisted of five different types of sorting algorithms). The results show the applicability and usefulness of roles of variables in AR, and illustrate that the C4.5 algorithm is a suitable decision tree classifier for our purpose. The limitations of the method are also discussed.

Categorizing variations of student-implemented sorting algorithms

“…The present study is a follow-up to the study discussed in Taherkhani (2011). The purpose of this study has been to validate the Aari system that uses the classification tree created from authentic students' submissions, and to see what kind of misconceptions students have with regard to sorting algorithms.…”

“…Moreover, when algorithms from different fields are covered and the number of recognizable algorithms increases, building a manual decision tree is not practical. We used the C4.5 algorithm to generate an automatic decision tree and reported the resulted tree in Taherkhani (2011). The tree was generated using a learning data set consisting of 209 implementations of the aforementioned sorting algorithms, i.e.…”

“…The steps illustrated in Figure 6 are described in more detail in our previous research (Taherkhani, 2011). The results of our experiment to recognize the students' implementations that are briefly presented in this section are obtained using the classification tree constructed in that research.…”

“…We have performed several enhancements to the original tool, and in its current state, the tool is able to detect the roles appearing in sorting algorithms very correctly. See Taherkhani (2011) for details on these enhancements. More information on how the tool works can be found in Bishop and Johnson (2005).…”

“…For example, we have categorized the algorithm depicted in Figure 1 as a variation of a Selection sort because of the steps taken in the two Figure 9. The classification tree generated by the C4.5 algorithm in our previous study (Taherkhani, 2011) for recognizing sorting algorithms.…”