Adversarial System Variant Approximation to Quantify Process Model Generalization

Theis, Julian; Darabi, Houshang

doi:10.1109/access.2020.3033450

Cited by 10 publications

(10 citation statements)

References 48 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Adversarial networks generalization. A recent generalization measure proposed in [10] uses sequence generative adversarial networks and the Metropolis-Hastings algorithm to sample unobserved trace variants based on the trace distribution of the input event log. The generalization is then computed as the harmonic mean between the fitness and the precision of the process model and the newly computed trace variants.…”

Section: Generalization Measuresmentioning

confidence: 99%

Generalization in Automated Process Discovery: A Framework based on Event Log Patterns

Reißner¹,

Armas-Cervantes²,

Rosa³

2022

Preprint

View full text Add to dashboard Cite

Automated process discovery is a tactical process mining capability that aims to describe the behavior of a business process recorded in an event log by means of a process model, in order achieve business process transparency. With a plethora of automated process discovery algorithms available, the importance of measuring the quality of a process model for a given event log has increased. One of the key quality aspects, generalization, is concerned with measuring the degree of overfitting of a process model w.r.t. an event log, since the recorded behavior is just an example of the true behavior of the underlying business process. Existing generalization measures exhibit several shortcomings that severely hinder their applicability in practice. For example, they assume the event log fully fits the discovered process model, and cannot deal with large real-life event logs and complex process models. More significantly, current measures neglect generalizations for clear patterns that demand a certain construct in the model. For example, a repeating sequence in an event log should be generalized with a loop structure in the model. We address these shortcomings by proposing a framework of measures that generalize a set of patterns discovered from an event log with representative traces and check the corresponding control-flow structures in the process model via their trace alignment. We instantiate the framework with a generalization measure that uses tandem repeats to identify repetitive patterns that are compared to the loop structures and a concurrency oracle to identify concurrent patterns that are compared to the parallel structures of the process model. In an extensive qualitative and quantitative evaluation using 74 log-model pairs using against two baseline generalization measures, we show that the proposed generalization measure consistently ranks process models that fulfil the observed patterns with generalizing control-flow structures higher than those which do not, while the baseline measures disregard those patterns. Further, we show that our measure can be efficiently computed for datasets two orders of magnitude larger than the largest dataset the baseline generalization measures can handle.

show abstract

Section: Generalization Measuresmentioning

confidence: 99%

Generalization in Automated Process Discovery: A Framework based on Event Log Patterns

Reißner¹,

Armas-Cervantes²,

Rosa³

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Given an event log, the process discovery problem consists of constructing a model that represents the behavior recorded in the log [1]. For example, suppose that L = [adeef 10 , addef, abbbcf 5 , abcf 20 , adef 20 , adefabcfadef 10 ] is an event log, with superscripts indicating multiplicity. Then L contains six distinct traces and 66 traces in total, with trace addef occurring only once, and traces abcf and adef both occurring twenty times.…”

Section: B Process Discoverymentioning

confidence: 99%

“…To resolve those tensions, we make use of a semiparametric bootstrap, borrowing initial ideas for estimating distributions of traces from Theis and Darabi [10] (see Section V). The semiparametric bootstrap assumes that the true population consists of elements similar but not necessarily identical to those in the original sample; another interpretation is that a semiparametric sample is a nonparametric sample with a certain amount of "noise" allowed.…”

Section: Log Samplingmentioning

confidence: 99%

“…The adversarial system variant approximation [10] uses the log's traces to train a sequence generative adversarial network (SGAN) that approximates the distribution of system traces, and then employs a sample of traces induced by the SGAN as a representation of the system's behavior, allowing measurement of generalization using standard approaches. Trained SGANs can also be incorporated into our bootstrap-based method to obtains a parametric bootstrap strategy, and we plan to explore this option in future work.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Bootstrapping Generalization of Process Models Discovered From Event Data

Polyvyanyy¹,

Moffat²,

García‐Bañuelos³

2021

Preprint

View full text Add to dashboard Cite

Process mining studies ways to derive value from process executions recorded in event logs of IT-systems, with process discovery the task of inferring a process model for an event log emitted by some unknown system. One quality criterion for discovered process models is generalization. Generalization seeks to quantify how well the discovered model describes future executions of the system, and is perhaps the least understood quality criterion in process mining. The lack of understanding is primarily a consequence of generalization seeking to measure properties over the entire future behavior of the system, when the only available sample of behavior is that provided by the event log itself. In this paper, we draw inspiration from computational statistics, and employ a bootstrap approach to estimate properties of a population based on a sample. Specifically, we define an estimator of the model's generalization based on the event log it was discovered from, and then use bootstrapping to measure the generalization of the model with respect to the system, and its statistical significance. Experiments demonstrate the feasibility of the approach in industrial settings.

show abstract

“…Moreover, the limitation ineluctably highlights the importance of obtaining trustworthy process models. Therefore, future work is anticipated to continue ongoing research efforts to assess the generalization of process models [6], [28] towards the discovery of process models that describe the realistic behavior of the underlying system.…”

mentioning

confidence: 99%

Masking Neural Networks Using Reachability Graphs to Predict Process Events

Theis,

Darabi

2021

Preprint

Self Cite

View full text Add to dashboard Cite

Decay Replay Mining is a deep learning method that utilizes process model notations to predict the next event. However, this method does not intertwine the neural network with the structure of the process model to its full extent. This paper proposes an approach to further interlock the process model of Decay Replay Mining with its neural network for next event prediction. The approach uses a masking layer which is initialized based on the reachability graph of the process model. Additionally, modifications to the neural network architecture are proposed to increase the predictive performance. Experimental results demonstrate the value of the approach and underscore the importance of discovering precise and generalized process models.

show abstract

Adversarial System Variant Approximation to Quantify Process Model Generalization

Cited by 10 publications

References 48 publications

Generalization in Automated Process Discovery: A Framework based on Event Log Patterns

Generalization in Automated Process Discovery: A Framework based on Event Log Patterns

Bootstrapping Generalization of Process Models Discovered From Event Data

Masking Neural Networks Using Reachability Graphs to Predict Process Events

Contact Info

Product

Resources

About