Debugging probabilistic programs

Nandi, Chandrakana; Grossman, Dan; Sampson, Adrian; Mytkowicz, Todd; McKinley, Kathryn S.

doi:10.1145/3088525.3088564

Cited by 10 publications

(5 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We use the Uncertain T programming language to implement random quantization. We choose it, because Uncertain T is sufficiently expressive and automates inference, and thus significantly simplifies our implementation [12], [42], [43]. The remainder of this section gives background on Uncertain T , the programming model that inspired and supports our technique, and describes our implementation.…”

Section: Methodsmentioning

confidence: 99%

High Five: Improving Gesture Recognition by Embracing Uncertainty

Tootaghaj,

Sampson,

Mytkowicz

et al. 2017

Preprint

Self Cite

View full text Add to dashboard Cite

Sensors on mobile devices-accelerometers, gyroscopes, pressure meters, and GPS-invite new applications in gesture recognition, gaming, and fitness tracking. However, programming them remains challenging because human gestures captured by sensors are noisy. This paper illustrates that noisy gestures degrade training and classification accuracy for gesture recognition in state-of-the-art deterministic Hidden Markov Models (HMM). We introduce a new statistical quantization approach that mitigates these problems by (1) during training, producing gesture-specific codebooks, HMMs, and error models for gesture sequences; and (2) during classification, exploiting the error model to explore multiple feasible HMM state sequences. We implement classification in Uncertain T , a probabilistic programming system that encapsulates HMMs and error models and then automates sampling and inference in the runtime. Uncertain T developers directly express a choice of applicationspecific trade-off between recall and precision at gesture recognition time, rather than at training time. We demonstrate benefits in configurability, precision, recall, and recognition on two data sets with 25 gestures from 28 people and 4200 total gestures. Incorporating gesture error more accurately in modeling improves the average recognition rate of 20 gestures from 34% in prior work to 62%. Incorporating the error model during classification further improves the average gesture recognition rate to 71%. As far as we are aware, no prior work shows how to generate an HMM error model during training and use it to improve classification rates.

show abstract

Section: Methodsmentioning

confidence: 99%

High Five: Improving Gesture Recognition by Embracing Uncertainty

Tootaghaj,

Sampson,

Mytkowicz

et al. 2017

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Having defined the statement-level transformations we now state a theorem about T β P [•] preserving continuity. As many applications may invoke inference at any point in the program [46,60], it is important that absolute continuity of each marginal hold at every point.…”

Section: Bringing It All Together: Full Program Transformationsmentioning

confidence: 99%

Continualization of Probabilistic Programs With Correction

Laurel

Misailovíc

2020

Programming Languages and Systems

View full text Add to dashboard Cite

Probabilistic Programming offers a concise way to represent stochastic models and perform automated statistical inference. However, many real-world models have discrete or hybrid discrete-continuous distributions, for which existing tools may suffer non-trivial limitations. Inference and parameter estimation can be exceedingly slow for these models because many inference algorithms compute results faster (or exclusively) when the distributions being inferred are continuous. To address this discrepancy, this paper presents Leios. Leios is the first approach for systematically approximating arbitrary probabilistic programs that have discrete, or hybrid discrete-continuous random variables. The approximate programs have all their variables fully continualized. We show that once we have the fully continuous approximate program, we can perform inference and parameter estimation faster by exploiting the existing support that many languages offer for continuous distributions. Furthermore, we show that the estimates obtained when performing inference and parameter estimation on the continuous approximation are still comparably close to both the true parameter values and the estimates obtained when performing inference on the original model.

show abstract

“…Verification and Analysis of Probabilistic Programs. Previous research proposed various techniques for statically analyzing and verifying properties of probabilistic programs, e.g., [6,11,14,25,31,33,37], including analyses that aim to help with debugging, e.g., [23,38,40]. In comparison, Storm presents a dynamic analysis that performs a heuristic search for smaller probabilistic program that reveal the same bugs.…”

Section: Related Workmentioning

confidence: 99%

“…The numerical and approximate nature of PP systems and implementation complexity make it hard to ensure their correctness, and subtle bugs can easily remain unnoticed [20,38,47]. Our recent study [10] showed that over 25% of all bugs in three popular systems are domain specific, including algorithmic, numerical, boundary condition, dimensional, and accuracy bugs.…”

Section: Introductionmentioning

confidence: 99%

“…If a failure occurs during the execution of a PP system, the developer typically has to figure out the source of the problem manually. This is not an easy task: while probabilistic programs are intuitive to write, they can be notoriously hard to analyze [26,38]. Probabilistic programs typically have a small number of lines of code, but they exercise many functionalities of the underlying PP system.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Storm: program reduction for testing and debugging probabilistic programming systems

Dutta

Zhang

Huang

et al. 2019

Proceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of

View full text Add to dashboard Cite

Probabilistic programming languages offer an intuitive way to model uncertainty by representing complex probability models as simple probabilistic programs. Probabilistic programming systems (PP systems) hide the complexity of inference algorithms away from the program developer. Unfortunately, if a failure occurs during the run of a PP system, a developer typically has very little support in finding the part of the probabilistic program that causes the failure in the system. This paper presents Storm, a novel general framework for reducing probabilistic programs. Given a probabilistic program (with associated data and inference arguments) that causes a failure in a PP system, Storm finds a smaller version of the program, data, and arguments that cause the same failure. Storm leverages both generic code and data transformations from compiler testing and domain-specific, probabilistic transformations. The paper presents new transformations that reduce the complexity of statements and expressions, reduce data size, and simplify inference arguments (e.g., the number of iterations of the inference algorithm). We evaluated Storm on 47 programs that caused failures in two popular probabilistic programming systems, Stan and Pyro. Our experimental results show Storm's effectiveness. For Stan, our minimized programs have 49% less code, 67% less data, and 96% fewer iterations. For Pyro, our minimized programs have 58% less code, 96% less data, and 99% fewer iterations. We also show the benefits of Storm when debugging probabilistic programs. CCS CONCEPTS • Software and its engineering → Software testing and debugging.

show abstract

Debugging probabilistic programs

Cited by 10 publications

References 23 publications

High Five: Improving Gesture Recognition by Embracing Uncertainty

High Five: Improving Gesture Recognition by Embracing Uncertainty

Continualization of Probabilistic Programs With Correction

Storm: program reduction for testing and debugging probabilistic programming systems

Contact Info

Product

Resources

About