Cross-Domain Evaluation of a Deep Learning-Based Type Inference System

Gruner, Bernd; Sonnekalb, Tim; Heinze, Thomas S.; Brust, Clemens-Alexander

doi:10.48550/arxiv.2208.09189

Cited by 1 publication

(7 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There are potential approaches where new types can be 1 https://gitlab.com/dlr-dw/type-inference recognized through additional static analysis, as demonstrated in HiTyper [15]. However, there is still the issue that data types that are rarely or not in the training dataset cannot be predicted [8]. Nevertheless, Novelty detection has not been actively pursued as a solution to the problem.…”

Section: Related Workmentioning

confidence: 99%

“…Following the pipeline of TIPICAL described in Section III-B, we present our experiments on the ManyTypes4Py and the CrossDomainTypes4Py datasets. Moreover, we expanded the scope of the experiments of the original papers [6], [8] to further study the effects of different software domains, and the unknown data types issues.…”

Section: Experiments and Evaluationmentioning

confidence: 99%

“…We use the CrossDomainTypes4Py [8] and ManyTypes4Py [11] datasets for our experiments. As described by Gruner et al [8], these contain a total of at least three different software domains.…”

Section: A Datasets and Domainsmentioning

confidence: 99%

“…Recent deep learning-based methods aim to overcome these issues and provide promising results [4]- [7]. However, these systems struggle with problems occurring in practical applications for example data types unknown to the system or source code from other software domains [8].…”

Section: Introductionmentioning

confidence: 99%

“…Furthermore, we investigate the impact of the data type frequencies and different software domains. For the evaluation, we use the recent datasets CrossDomainTypes4Py [8] and ManyTypes4Py [11]. In order to ensure the reproducibility of our experiments, we make our experimental pipeline publicly available 1 .…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

TIPICAL - Type Inference for Python In Critical Accuracy Level

Elkobi,

Gruner,

Sonnekalb

et al. 2023

2023 IEEE/ACIS 21st International Conference on Software Engineering Research, Management and Applications (SERA)

View full text Add to dashboard Cite

Type inference methods based on deep learning are becoming increasingly popular as they aim to compensate for the drawbacks of static and dynamic analysis approaches, such as high uncertainty. However, their practical application is still debatable due to several intrinsic issues such as code from different software domains will involve data types that are unknown to the type inference system.In order to overcome these problems and gain high-confidence predictions, we thus present TIPICAL, a method that combines deep similarity learning with novelty detection. We show that our method can better predict data types in high confidence by successfully filtering out unknown and inaccurate predicted data types and achieving higher F1 scores to the state-of-the-art type inference method Type4Py. Additionally, we investigate how different software domains and data type frequencies may affect the results of our method.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Experiments and Evaluationmentioning

confidence: 99%