Automatic Classification of Web Images as UML Static Diagrams Using Machine Learning Techniques

Abstract

Our purpose in this research is to develop a method to automatically and efficiently classify web images as Unified Modeling Language (UML) static diagrams, and to produce a computer tool that implements this function. The tool receives a bitmap file (in different formats) as an input and communicates whether the image corresponds to a diagram. For pragmatic reasons, we restricted ourselves to the simplest kinds of diagrams that are more useful for automated software reuse: computer-edited 2D representations of static diagrams. The tool does not require that the images are explicitly or implicitly tagged as UML diagrams. The tool extracts graphical characteristics from each image (such as grayscale histogram, color histogram and elementary geometric forms) and uses a combination of rules to classify it. The rules are obtained with machine learning techniques (rule induction) from a sample of 19,000 web images manually classified by experts. In this work, we do not consider the textual contents of the images. Our tool reaches nearly 95% of agreement with manually classified instances, improving the effectiveness of related research works. Moreover, using a training dataset 15 times bigger, the time required to process each image and extract its graphical features (0.680 s) is seven times lower.

Automatic Classification of UML Class Diagrams Using Deep Learning Technique: Convolutional Neural Network

“…A recent and substantial study of UML models stored in different formats in GitHub, combining a lot of manual work and automatic processing recognized 93,596 UML models from 24,717 different repositories, of which 61.8% (57,822) are images, the rest being files with extensions, XML or UML [24]. These data confirm that the existence of interesting and potentially useful information in repositories are not up to the mark which makes it difficult to access and reuse; this also gives the idea that a huge proportion of such models are stored as images [10].…”

“…They proposed an approach of simpler deep architectures in combination with low shot learning to reduce the learning parameters for classifying UML Class and Sequence diagrams. Moreno et al [10] developed a method to automatically and efficiently classify web images as Unified Modeling Language (UML) static diagrams and to produce a computer tool that whether the image corresponds to a diagram. The tool selects graphical characteristics from every image (color histogram, grayscale histogram, and elementary geometric forms) and uses a combination of rules obtained using Machine Learning techniques to classify the images.…”

“…Karasneh et al [16] have used a dataset of 200 images and got an accuracy of 89%, Ho-Quang et al [3] got an accuracy of 91.2% on the dataset of 1300 images and they used the SVM algorithm. Ho-Quang et al [3] got up to 93.2% accuracy on 1300 images in their dataset, Valentin Mareno et al [10] developed a model called image-to-model (i2m) using a dataset of 18,899 images and got an accuracy of 94.4%. Ott Jordan et al [30] have used the technique of low-short learning which is very useful when the dataset is very small and combined with the Convolutional Neural Network technique on the dataset compressing of 11,319 images to classify UML class diagrams from sequence diagram, and they claimed accuracy of more than 92%.…”

“…However, such private repositories have few drawbacks as they support only support the native file format of the CASE tools used by them and they do not allow any open access to the models generated by some other tools. On the other hand, for the public side, France et al proposed ReMoDD, a pioneering repository for model-driven development [9,10]. However, those models cannot be easily retrieved because the models are stored as files that have to be downloaded and then opened with a compatible CASE tool otherwise, they are pure PDF files.…”

“…Therefore, we can conclude although several initiatives have been taken to promote and support the model adoption, we are still far apart from a proper and concrete adoption of such model repositories [12]. Automatic classification of web images was done using a Machine Learning approach [10], which helped in improving the efficacy of common web search engines like Google Images. The approach also proved to be useful to search the repositories when metadata is not available.…”