A probabilistic evaluation procedure for process model matching techniques

Kuss, Elena; Leopold, Henrik; Aa, Han van der; Stuckenschmidt, Heiner

doi:10.1016/j.datak.2018.04.008

Cited by 15 publications

(7 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Business process models can be compared to evaluate its structure [17,18] and behavior [19,20] that have similar functions [21]. The comparison is called a business process similarity [22].…”

Section: Introductionmentioning

confidence: 99%

Novel Parallel Business Process Similarity Methods Based on Weighted-Tree Declarative Pattern Models

Wahyuni¹,

Sungkono²,

Sarno³

2019

IJIES

View full text Add to dashboard Cite

Existing methods, such as Graph Edit Distance (GED) and Cosine measure, still have drawback in obtaining similarity of parallel relationships by neglecting the control-flow patterns, i.e. AND, OR, and XOR. Since AND > OR > XOR, the similarity value of AND versus OR is greater than XOR versus OR and AND versus XOR. This paper proposes two new similarity methods, Tree Declarative Pattern Edit Distance (TPED) and Cosine-Tree Declarative Pattern (Cosine-TDP). They provides value to the control-flow pattern so the value of similarity can be seen more differently. The new methods utilize tree model of the declarative pattern. The results show that the proposed methods are better at differentiating parallel relationships than the existing methods, GED and Cosine measure. In obtaining AND versus OR, XOR versus OR, and AND versus XOR, TPED obtained 0.821, 0.811, and 0.78 while Consine-TDP obtained 0.834, 0.826, and 0.693. Meanwhile, GED obtained 1 for all parallel relationships whereas Cosine measure obtained 0.02, 0.08, and 0.04.

show abstract

Section: Introductionmentioning

confidence: 99%

Novel Parallel Business Process Similarity Methods Based on Weighted-Tree Declarative Pattern Models

Wahyuni¹,

Sungkono²,

Sarno³

2019

IJIES

View full text Add to dashboard Cite

show abstract

“…F1 score is an established performance measure to quantitatively evaluate the accuracy of process model matching techniques [6]. By definition, the F1 score is a harmonic mean of Precision and Recall, which is a conservative mean that exhibits a higher value when both precision and recall are high.…”

Section: Resultsmentioning

confidence: 99%

A Process Model Collection and Gold Standard Correspondences for Process Model Matching

et al. 2019

View full text Add to dashboard Cite

Business process models are the conceptual models to depict the workflow of an organization. Process model matching (PMM) refers to the automatic identification of corresponding activities between a pair of process models that show similar or the same behavior. During the last few years, PMM has received much of the researchers' attention due to its wide range of applications, such as clone detection and harmonization of process models. Consequently, a plethora of PMM techniques has been developed. In order to evaluate the effectiveness of these techniques, experts have developed three benchmark datasets, formally called PMMC'15 datasets. Furthermore, the process models in the datasets have been converted into OAEI'17 ontologies. These resources are a valuable asset for the PMM community to evaluate process model matching techniques. However, these resources (PMMC'15 and OAEI'17) are limited to fewer models and a handful collection of corresponding activities among these models that may not be sufficient to rigorously evaluate the PMM techniques. To fill this gap, this paper provides a large, diverse, and a carefully handcrafted collection of process models, along with their benchmark correspondences. The process model collection and benchmark correspondences between these models are freely available for the community [1]. Our newly developed dataset, together with the existing resources, can be used for a thorough evaluation of PMM techniques, especially in the context of the vocabulary mismatch problem. At last, we have evaluated the characteristics of our dataset by a series of experiments while involving widely used similarity measures in PMM research. The results reveal that our dataset is larger, diverse, and challenging as compared to existing datasets in the PMM domain.

show abstract

“…There are three publicly available datasets that have been widely used to evaluate the effectiveness of PMM techniques [4], [13]. These datasets include real-world process models from three different domains: university admission, registration of newborns, and asset management.…”

Section: B Establishing the Problemmentioning

confidence: 99%

“…to use string matching techniques for PMM [11]. Typically, the effectiveness of these techniques is evaluated in terms of three measures, Precision, Recall, and F1 scores, using benchmark datasets from the process model matching contests [11]- [13]. In this study, we have synthesized these available benchmark datasets to reveal two interrelated problems, a) the F1 score achieved by PMM is inflated due to the presence of trivial corresponding pairs, and b) the three measures merely establish the effectiveness of a PMM technique at surface-level and do not provide deeper insights into PMM techniques.…”

Section: Introductionmentioning

confidence: 99%

Deep Analysis of Process Model Matching Techniques

et al. 2020

View full text Add to dashboard Cite

Process Model Matching (PMM) aims to automatically identify corresponding activities from two process models that exhibit similar behaviors. Recognizing the diverse applications of process model matching, several techniques have been proposed in the literature. Typically, the effectiveness of these matching techniques has been evaluated using three widely used performance measures, Precision, Recall, and F1 score. In this study, we have established that the values of these three measures for each dataset do not provide deeper insights into the capabilities of the matching techniques. To that end, we have made three significant contributions. Firstly, we have enhanced four benchmark datasets by classifying their corresponding activities into three sub-types. The enhanced datasets can be used for surface-level evaluation, as well as a deeper evaluation of matching techniques. Secondly, we have conducted a systematic search of the literature to identify an extensive set of 27 matching techniques and subsequently proposed a taxonomy for these matching techniques. Finally, we have performed 432 experiments to evaluate the effectiveness of all the matching techniques, and key observations about the effectiveness of the techniques are presented. INDEX TERMS Business process management, developing benchmarks, taxonomy, text matching.

show abstract

A probabilistic evaluation procedure for process model matching techniques

Cited by 15 publications

References 42 publications

Novel Parallel Business Process Similarity Methods Based on Weighted-Tree Declarative Pattern Models

Novel Parallel Business Process Similarity Methods Based on Weighted-Tree Declarative Pattern Models

A Process Model Collection and Gold Standard Correspondences for Process Model Matching

Deep Analysis of Process Model Matching Techniques

Contact Info

Product

Resources

About