VyPR2: A Framework for Runtime Verification of Python Web Services

Dawes, Joshua Heneage; Reger, Giles; Franzoni, G.; Pfeiffer, A.; Govi, G.

doi:10.1007/978-3-030-17465-1_6

Cited by 7 publications

(4 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The application of the first prototype of VyPR to the CMS Conditions Upload Service yielded two instances of interesting behaviour [10]. One was an unexpectedly large amount of time taken by one query, and another was that the times taken by many repetitions of a particular query seemed to be inversely proportional to the time between the queries.…”

Section: Previous Resultsmentioning

confidence: 99%

See 1 more Smart Citation

Analysis Tools for the VyPR Performance Analysis Framework for Python

et al. 2020

Self Cite

View full text Add to dashboard Cite

VyPR (http://pyvypr.github.io/home/) is a framework being developed with the aim of automating as much as possible the performance analysis of Python programs. To achieve this, it uses an analysis-by-specification approach; developers specify the performance requirements of their programs (without any modifications of the source code) and such requirements are checked at runtime. VyPR then provides tools which allow developers to perform detailed analyses of the performance of their code. Such analyses can include determining the common paths taken to reach badly performing parts of code, deciding whether a single path through code led to variations in time taken by future observations, and more. This paper describes the developments that have taken place in the past year on VyPR’s analysis tools to yield a Python shell-based analysis library, and a web-based application. It concludes by demonstrating the use of the analysis tools on the CMS Experiment’s Conditions Upload service.

show abstract

Section: Previous Resultsmentioning

confidence: 99%

“…Given any specification that is written in PyCFTL, VyPR can automatically instrument the program under scrutiny (so that a conservative amount of data is taken at runtime) and efficiently monitor for agreement with the specification at runtime [9,10]. Instrumentation is performed by:…”

Section: Instrumentation and Monitoringmentioning

confidence: 99%

Analysis Tools for the VyPR Performance Analysis Framework for Python

et al. 2020

Self Cite

View full text Add to dashboard Cite

show abstract

“…An originality is the notion of severity, explaining by how much a timing constraint is violated (which shares similarities with robustness [DM10]). The approach is implemented into VyPR2 [Daw+19]. A main difference with our approach is that [DR19] targets the explanation of one particular run violation, whereas we seek arbitrary exemplifications of a property (both positive and negative), independently of a run.…”

Section: Related Workmentioning

confidence: 99%

Exemplifying parametric timed specifications over signals with bounded behavior

André,

Waga,

Urabe

et al. 2022

Preprint

View full text Add to dashboard Cite

Specifying properties can be challenging work. In this paper, we propose an automated approach to exemplify properties given in the form of automata extended with timing constraints and timing parameters, and that can also encode constraints over real-valued signals. That is, given such a specification and given an admissible automaton for each signal, we output concrete runs exemplifying real (or impossible) runs for this specification. Specifically, our method takes as input a specification, and a set of admissible behaviors, all given as a subclass of rectangular hybrid automata, namely timed automata extended with arbitrary clock rates, signal constraints, and timing parameters. Our method then generates concrete runs exemplifying the specification.

show abstract

“…The deployed SA systems receive many queries that normally mismatch pre-defined templates, making it challenging to detect biased predictions at runtime. Runtime verification (RV) is the process of checking whether each run of a system satisfies a given property [5]. To the best of our knowledge, there is no tool that can uncover biased prediction of an SA system at runtime.…”

Section: Introductionmentioning

confidence: 99%

BiasRV: Uncovering Biased Sentiment Predictions at Runtime

Yang,

Asyrofi,

2021

Preprint

View full text Add to dashboard Cite

Sentiment analysis (SA) systems, though widely applied in many domains, have been demonstrated to produce biased results. Some research works have been done in automatically generating test cases to reveal unfairness in SA systems, but the community still lacks tools that can monitor and uncover biased predictions at runtime. This paper fills this gap by proposing BiasRV, the first tool to raise an alarm when a deployed SA system makes a biased prediction on a given input text. To implement this feature, BiasRV dynamically extracts a template from an input text and from the template generates gender-discriminatory mutants (semanticallyequivalent texts that only differ in gender information). Based on popular metrics used to evaluate the overall fairness of an SA system, we define distributional fairness property for an individual prediction of an SA system. This property specifies a requirement that for one piece of text, mutants from different gender classes should be treated similarly as a whole. Verifying the distributional fairness property causes much overhead to the running system. To run more efficiently, BiasRV adopts a two-step heuristic: (1) sampling several mutants from each gender and checking if the system predicts them as of the same sentiment, (2) checking distributional fairness only when sampled mutants have conflicting results. Experiments show that compared to directly checking the distributional fairness property for each input text, our two-step heuristic can decrease overhead used for analyzing mutants by 73.81% while only resulting in 6.7% of biased predictions being missed. Besides, BiasRV can be used conveniently without knowing the implementation of SA systems. Future researchers can easily extend BiasRV to detect more types of bias, e.g. race and occupation. The demo video for BiasRV can be viewed at https://youtu.be/WPe4Ml77d3U and the source code can be found at https://github.com/soarsmu/BiasRV.

show abstract

VyPR2: A Framework for Runtime Verification of Python Web Services

Cited by 7 publications

References 18 publications

Analysis Tools for the VyPR Performance Analysis Framework for Python

Analysis Tools for the VyPR Performance Analysis Framework for Python

Exemplifying parametric timed specifications over signals with bounded behavior

BiasRV: Uncovering Biased Sentiment Predictions at Runtime

Contact Info

Product

Resources

About