Crowdsourced user interface testing for multimedia applications

Vliegendhart, Raynor; Dolstra, Eelco; Pouwelse, Johan

doi:10.1145/2390803.2390813

Cited by 11 publications

(7 citation statements)

References 2 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It has been applied to usability testing [30], [154]- [159], performance testing [160], GUI testing [87], [161], test case generation [166], [167], and the oracle problem [88]. We discuss each of these below: 1) Usability Testing: Traditional usability testing is labour-intensive and can be expensive and time consuming [155].…”

Section: Crowdsourcing For Software Testingmentioning

confidence: 99%

See 1 more Smart Citation

A survey of the use of crowdsourcing in software engineering

Mao

Capra

Harman

et al. 2017

Journal of Systems and Software

304

231

View full text Add to dashboard Cite

Crowdsourcing can be used to support software engineering activities and research into these activities. In this paper we provide a comprehensive survey of the use of crowdsourcing to support software engineering activities (Crowdsourced Software Engineering), seeking to cover all literature on this topic. We describe the software engineering domains, tasks and applications for crowdsourcing and the platforms and stakeholders involved in realising Crowdsourced Software Engineering solutions. We also expose trends, issues and opportunities for Crowdsourced Software Engineering.Please cite as: Ke Mao, Licia Capra, Mark Harman and Yue Jia. A Survey of the Use of Crowdsourcing in SoftwareEngineering.

show abstract

Section: Crowdsourcing For Software Testingmentioning

confidence: 99%

“…Vliegendhart et al [161] first proposed GUI testing for multimedia applications. Crowd testers were recruited from Amazon Mechanical Turk.…”

Section: Crowdsourcing For Software Testingmentioning

confidence: 99%

A survey of the use of crowdsourcing in software engineering

Mao

Capra

Harman

et al. 2017

Journal of Systems and Software

304

231

View full text Add to dashboard Cite

show abstract

“…Table 1 depicts the references identified per type of testing. [3], [43], [44], [45], [46], [47], [48] Non-Functional Testing [32], [43], [44], (performance); [49] (vulnerability); [50] (privacy) Validation and Acceptance Testing [11], [34], [35], [36], Usability Testing/ User Experience [35], [41], [51], [52], [53], [54], [55], [56], [57] Quality of Experience [26], [58], [59], [60], [61], [62] Research in functional and verification testing demonstrated that even complex testing tasks such as the verification of cross-browser issues [46] or the reproduction of context-sensitive app crashes [45] are possible to be tested with the crowd. In this vein, also non-functional testing such as performance testing [32] is possible.…”

Section: Application Of Crowdsourced Software Testingmentioning

confidence: 99%

Given Enough Eyeballs, all Bugs are Shallow - A Literature Review for the Use of Crowdsourcing in Software Testing

Leicht¹

2018

Proceedings of the 51st Hawaii International Conference on System Sciences

View full text Add to dashboard Cite

Over the last years, the use of crowdsourcing has gained a lot of attention in the domain of software engineering. One key aspect of software development is the testing of software. Literature suggests that crowdsourced software testing (CST) is a reliable and feasible tool for manifold kinds of testing. Research in CST made great strides; however, it is mostly unstructured and not linked to traditional software testing practice and terminology. By conducting a literature review of traditional and crowdsourced software testing literature, this paper delivers two major contributions. First, it synthesizes the fields of crowdsourcing research and traditional software testing. Second, the paper gives a comprehensive overview over findings in CSTresearch and provides a classification into different software testing types.

show abstract

“…Previous work on crowdsourced software testing has formulated the test design problem as one to be outsourced to the crowd. For example, Dolstra et al [28] and Vliegendhart et al [29] demonstrated the usefulness of using Amazon Mechanical Turk workforce to perform continuous GUI testing.…”

Section: Related Workmentioning

confidence: 99%

Crowd intelligence enhances automated mobile testing

Mao

Harman

Jia

2017

2017 32nd IEEE/ACM International Conference on Automated Software Engineering (ASE)

View full text Add to dashboard Cite

We show that information extracted from crowdbased testing can enhance automated mobile testing. We introduce POLARIZ, which generates replicable test scripts from crowd-based testing, extracting cross-app 'motif' events: automatically-inferred reusable higher-level event sequences composed of lower-level observed event actions. Our empirical study used 434 crowd workers from Mechanical Turk to perform 1,350 testing tasks on 9 popular Google Play apps, each with at least 1 million user installs. The findings reveal that the crowd was able to achieve 60.5% unique activity coverage and proved to be complementary to automated search-based testing in 5 out of the 9 subjects studied. Our leave-one-out evaluation demonstrates that coverage attainment can be improved (6 out of 9 cases, with no disimprovement on the remaining 3) by combining crowdbased and search-based testing.

show abstract

Crowdsourced user interface testing for multimedia applications

Cited by 11 publications

References 2 publications

A survey of the use of crowdsourcing in software engineering

A survey of the use of crowdsourcing in software engineering

Given Enough Eyeballs, all Bugs are Shallow - A Literature Review for the Use of Crowdsourcing in Software Testing

Crowd intelligence enhances automated mobile testing

Contact Info

Product

Resources

About