Source Code Plagiarism Detection in Academia with Information Retrieval: Dataset and the Observation

Abstract

Source code plagiarism is an emerging issue in computer science education. As a result, a number of techniques have been proposed to handle this issue. However, comparing these techniques may be challenging, since they are evaluated with their own private dataset(s). This paper contributes in providing a public dataset for comparing these techniques. Specifically, the dataset is designed for evaluation with an Information Retrieval (IR) perspective. The dataset consists of 467 source code files, covering seven introductory programming assessment tasks. Unique to this dataset, both intention to plagiarise and advanced plagiarism attacks are considered in its construction. The dataset's characteristics were observed by comparing three IR-based detection techniques, and it is clear that most IR-based techniques are less effective than a baseline technique which relies on Running-Karp-Rabin Greedy-String-Tiling, even though some of them are far more time-efficient.

Explanation in Code Similarity Investigation

“…Recall is the proportion of suspected program pairs that are copied to all copied pairs. To perform this comparison we used the IR-Plag data set [52]. This data set is based on seven initial Java programs that cover introductory programming materials (output, input, branching, looping, function, array, and matrix).…”

TF-IDF Inspired Detection for Cross-Language Source Code Plagiarism and Collusion

“…The introductory dataset was created by rewriting seven original code files in a Java dataset [28] adapted from Liang's book [31] to eight other programming languages: C++, C#, Javascript, Kotlin, Pascal, Python, R, and Scala. These code files cover seven introductory materials: the output, input, branching, looping, function, array, and matrix.…”

“…The same-language dataset is the Java dataset used for creating the introductory dataset [28]. Each original code file was copied and modified according to the last six levels of similarity disguise taxonomy [16] (the first level was excluded, as it depicted no disguises) by nine teaching assistants.…”

Initial Suspicion on Detecting Code Plagiarism and Collusion in Academia: Case Study of Algorithm and Data Structure Courses

“…Both techniques of raising initial suspicion require lecturer's knowledge about student academic performance. We cannot therefore use publicly available data sets [21], [22], [23], [24] as they have no such information. We create our own data set just for this purpose.…”