We introduce Sapienz, an approach to Android testing that uses multi-objective search-based testing to automatically explore and optimise test sequences, minimising length, while simultaneously maximising coverage and fault revelation. Sapienz combines random fuzzing, systematic and search-based exploration, exploiting seeding and multi-level instrumentation. Sapienz significantly outperforms (with large effect size) both the state-of-the-art technique Dynodroid and the widely-used tool, Android Monkey, in 7/10 experiments for coverage, 7/10 for fault detection and 10/10 for fault-revealing sequence length. When applied to the top 1,000 Google Play apps, Sapienz found 558 unique, previously unknown crashes. So far we have managed to make contact with the developers of 27 crashing apps. Of these, 14 have confirmed that the crashes are caused by real faults. Of those 14, six already have developer-confirmed fixes. CCS Concepts •Software and its engineering → Software testing and debugging; Search-based software engineering;

Crowdsourcing can be used to support software engineering activities and research into these activities. In this paper we provide a comprehensive survey of the use of crowdsourcing to support software engineering activities (Crowdsourced Software Engineering), seeking to cover all literature on this topic. We describe the software engineering domains, tasks and applications for crowdsourcing and the platforms and stakeholders involved in realising Crowdsourced Software Engineering solutions. We also expose trends, issues and opportunities for Crowdsourced Software Engineering.Please cite as: Ke Mao, Licia Capra, Mark Harman and Yue Jia. A Survey of the Use of Crowdsourcing in SoftwareEngineering.

We describe the deployment of the Sapienz Search Based Software Engineering (SBSE) testing system. Sapienz has been deployed in production at Facebook since September 2017 to design test cases, localise and triage crashes to developers and to monitor their fixes. Since then, running in fully continuous integration within Facebook's production development process, Sapienz has been testing Facebook's Android app, which consists of millions of lines of code and is used daily by hundreds of millions of people around the globe. We continue to build on the Sapienz infrastructure, extending it to provide other software engineering services, applying it to other apps and platforms, and hope this will yield further industrial interest in and uptake of SBSE (and hybridisations of SBSE) as a result. This paper was written to accompany the keynote by Mark Harman at the 10 th Symposium on Search-Based Software Engineering (SSBSE 2018), Montpellier September 8-10, 2018. The paper represents the work of all the authors in realising the deployment of search based approaches to large-scale software engineering at Facebook. Author name order is alphabetical; the order is thus not intended to denote any information about the relative contribution of each author.

We report our experience with SAPFIX: the first deployment of automated end-to-end fault fixing, from test case design through to deployed repairs in production code 1. We have used SAPFIX at Facebook to repair 6 production systems, each consisting of tens of millions of lines of code, and which are collectively used by hundreds of millions of people worldwide.

No abstract

No abstract

We show that information extracted from crowdbased testing can enhance automated mobile testing. We introduce POLARIZ, which generates replicable test scripts from crowd-based testing, extracting cross-app 'motif' events: automatically-inferred reusable higher-level event sequences composed of lower-level observed event actions. Our empirical study used 434 crowd workers from Mechanical Turk to perform 1,350 testing tasks on 9 popular Google Play apps, each with at least 1 million user installs. The findings reveal that the crowd was able to achieve 60.5% unique activity coverage and proved to be complementary to automated search-based testing in 5 out of the 9 subjects studied. Our leave-one-out evaluation demonstrates that coverage attainment can be improved (6 out of 9 cases, with no disimprovement on the remaining 3) by combining crowdbased and search-based testing.

R obots are widely used for many repetitive tasks. Why not software testing? Robotic Testing could give testers a completely new form of 'blackbox' testing, that is inherently more black box than anything witnessed previously. This article provides a comparison between the state-of-the-art simulated-based mobile test automation and our proposed robotic mobile testing. We give with scenarios for which robotic testing is beneficial (even essential), introducing a robotic mobile device test generator, Axiz. We illustrate the application of Axiz to a popular Google Calculator app. Automated Robotic Mobile Testing We advocate a Ropbotic Tetsing appropach to address the profound shift we are currently witnessing, from desktop to mobile computation 1. This is a trend which is projected to gather pace 2 , accelerated by a concomitant shift from desktop to mobile ownership. Automated software testing is needed more than ever in this emerging mobile world, yet we may need to rethink some of the principles of software testing, even fundamentals, such as what it means to be truly 'black box' when testing. Mobile devices enable rich user interaction inputs such as gestures via touch screens and various signals via sensors (GPS, accelerometer, barometer, NFC, etc.).