Program Sketching by Automatically Generating Mocks from Tests

Abstract

Sketch is a popular program synthesis tool that solves for unknowns in a sketch or partial program. However, while Sketch is powerful, it does not directly support modular synthesis of dependencies, potentially limiting scalability. In this paper, we introduce Sketcham, a new technique that modularizes a regular sketch by automatically generating mocks—functions that approximate the behavior of complete implementations—from the sketch’s test suite. For example, if the function f originally calls g, Sketcham creates a mock $$g_m$$ g m from g’s tests and augments the sketch with a version of f that calls $$g_m$$ g m . This change allows the unknowns in f and g to be solved separately, enabling modular synthesis with no extra work from the Sketch user. We evaluated Sketcham on ten benchmarks, performing enough runs to show at a 95% confidence level that Sketcham improves median synthesis performance on six of our ten benchmarks by a factor of up to 5$$\times $$ × compared to plain Sketch, including one benchmark that times out on Sketch, while exhibiting similar performance on the remaining four. Our results show that Sketcham can achieve modular synthesis by automatically generating mocks from tests.

Mimicking Production Behavior with Generated Mocks

“…This approach has been used by Saff et al [8] through system test executions, and Fazzini et al [34] to generate mocks for mobile applications. Bragg et al [60] use the test suite of Sketch programs to generate mocks in order to modularize program synthesis. Mocks have also been generated in the context of cloud computing [61], such as for the emulation of infrastructure by MockFog [15].…”