Abstract-and-Compare: A Family of Scalable Precision Measures for Automated Process Discovery

Abstract

Automated process discovery techniques allow us to extract business process models from event logs. The quality of process models discovered by these techniques can be assessed with respect to various quality criteria related to simplicity and accuracy. One of these criteria, namely precision, captures the extent to which the behavior allowed by a discovered process model is observed in the log. While numerous measures of precision have been proposed in the literature, a recent study has shown that none of them fulfils a set of five axioms that capture intuitive properties behind the concept of precision. In addition, several existing precision measures suffer from scalability issues when applied to models discovered from real-life event logs. This paper presents a versatile framework for defining precision measures based on behavior abstractions. The key idea is that a precision measure can be defined by three ingredients: a function that abstracts a process model (e.g. as a transition system), a function that does the same for an event log, and a function that compares the behavior abstraction of the model with that of the log. We show empirically that different instances of this framework allow us to strike different tradeoffs between scalability and sensitivity. We also show that two instances of the framework based on lossless abstraction functions yield a precision measure that fulfils all the above-mentioned axioms. 1 A third accuracy criterion in automated process discovery is generalization: the extent to which the process model captures behavior that, while not observed in the log, is implied by it.

Metaheuristic Optimization for Automated Business Process Discovery

“…We implemented our approach as a Java command-line application 4 using Split Miner as the underlying automated process discovery approach and Markovian accuracy Fscore as the objective function (cf. Section 3.4).…”

“…Among the existing measures of fitness and precision we selected the Markovian fitness and precision defined in [5] (boolean function variant, order k = 5). The rationale for this choice is that these measures of fitness and precision are the fastest to compute among state-of-the-art measures [4,5]. Furthermore, the Markvovian fitness (precision) provides a feedback that tells us what edges could be added to (removed from) the DFG to improve the fitness (precision).…”

Monotone Conformance Checking for Partially Matching Designed and Observed Processes

“…In this section, we experiment with a synthetic event log and a set of corresponding process models described in [14], [15]. The event log is defined as this set of traces: A, B, D, E, I , A, C, D, G, H, F, I , A, C, G, D, H, F, I , A, C, H, D, F, I , A, C, D, H, F, I }.…”

“…The table shows our rankings (last two columns) and compares them with the rankings of other conformance techniques calculated for the same models and log in [14], [15]. Concretely, these techniques are used (refer to columns 2-8 in the table): Set difference (SD) [16], Negative events (NE) [17], Escaping edges (ETC) [18], Alignment-based ETC precision (ETCa) [19], Projected conformance checking (PCC) [20], Anti-alignment precision (AA) [14], [21], and k-order Markovian abstractions (MAP k ) [15]. Greater values of k for the latter technique correspond to less abstraction in the encodings of models and logs and, thus, more "precise" precision measurements.…”

“…Finally, note that in contrast to the anti-alignment precision, using both our precision measures, i.e., EB and EB τ , allows for a more complete analysis by differentiating between Flower model and Round robin model. Figure 6: Artificial process models from [14], [15].…”