FAUSTA: Scaling Dynamic Analysis with Traffic Generation at WhatsApp

Mao, Ke; Kapus, Timotej; Petrou, Lambros; Hajdu, Ákos; Marescotti, Matteo; Löscher, Andreas; Harman, Mark; Distefano, Dino

doi:10.1109/icst53961.2022.00036

Cited by 7 publications

(1 citation statement)

References 49 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Such behaviours can be deemed to be incorrect behaviour, irrespective of the input used to test the system under test. At Meta Platforms, we have previously used an implicit oracle in our work on simulation-based testing using Sapienz [6] and FAUSTA [31] for example. The implicit oracle is a powerful way to overcome the oracle problem for testing problems associated with reliability, both client-side application reliability (in the case of Sapienz) and server-side reliability (in the case of FAUSTA).…”

Section: Related Workmentioning

confidence: 99%

Simulation-Driven Automated End-to-End Test and Oracle Inference

Tuli¹,

Bojarczuk²,

Gucevska³

et al. 2023

Preprint

View full text Add to dashboard Cite

This is the first work to report on inferential testing at scale in industry. Specifically, it reports the experience of automated testing of integrity systems at Meta. We built an internal tool called ALPACAS for automated inference of end-toend integrity tests. Integrity tests are designed to keep users safe online by checking that interventions take place when harmful behaviour occurs on a platform. ALPACAS infers not only the test input, but also the oracle, by observing production interventions to prevent harmful behaviour. This approach allows Meta to automate the process of generating integrity tests for its platforms, such as Facebook and Instagram, which consist of hundreds of millions of lines of production code. We outline the design and deployment of ALPACAS, and report results for its coverage, number of tests produced at each stage of the test inference process, and their pass rates. Specifically, we demonstrate that using ALPACAS significantly improves coverage from a manual test design for the particular aspect of integrity end-to-end testing it was applied to. Further, from a pool of 3 million data points, ALPACAS automatically yields 39 production-ready end-to-end integrity tests. We also report that the ALPACAS-inferred test suite enjoys exceptionally low flakiness for end-to-end testing with its average in-production pass rate of 99.84%.

show abstract

Section: Related Workmentioning

confidence: 99%