BigDebug

Abstract

Developers use cloud computing platforms to process a large quantity of data in parallel when developing big data analytics. Debugging the massive parallel computations that run in today’s data-centers is time consuming and error-prone. To address this challenge, we design a set of interactive, real-time debugging primitives for big data processing in Apache Spark, the next generation data-intensive scalable cloud computing platform. This requires re-thinking the notion of step-through debugging in a traditional debugger such as , because pausing the entire computation across distributed worker nodes causes significant delay and naively inspecting millions of records using a watchpoint is too time consuming for an end user. First, BIGDEBUG’s simulated breakpoints and on-demand watchpoints allow users to selectively examine distributed, intermediate data on the cloud with little overhead. Second, a user can also pinpoint a crash-inducing record and selectively resume relevant sub-computations after a quick fix. Third, a user can determine the root causes of errors (or delays) at the level of individual records through a fine-grained data provenance capability. Our evaluation shows that BIGDEBUG scales to terabytes and its record-level tracing incurs less than 25% overhead on average. It determines crash culprits orders of magnitude more accurately and provides up to 100% time saving compared to the baseline replay debugger. The results show that BIGDEBUG supports debugging at interactive speeds with minimal performance impact.

Debugging data flows in reactive programs

“…While the rst is arguably not desirable for reactive systems, the noti cation property might be a good addition to RxFiddle. BIGDEBUG [26], a debugging solution for systems like Spark [52], introduces simulated breakpoints for this purpose. When a simulated breakpoint is reached, the execution resumes immediately and the required lineage information of the breakpoint is collected in a new independent process.…”

Out-Of-Place debugging: a debugging architecture to reduce debugging interference

“…In order to deal with non-deterministic inputs, a partial order of variable accesses or events has to be stored in the trace to be able to reproduce the program concurrent behaviour. To make replay debugging scalable, several debuggers combine trace recording with checkpoint-based debugging, in which snapshots of the application are taken at certain periods of time to limit the size of the trace that needs to be stored [1,16].…”

“…As such, it may not be convenient to let a temperature monitoring application crash, and retrieve a log or a trace of the execution afterward. In some cases, stopping the execution of the program may also lead to information loss [16].…”

“…As previously explained, offline debugging techniques can be tedious when logs are used, or they introduce a significant overhead in the case of record and replay [16]. More importantly, they require many debugging cycles to find the root cause of a bug since contextual information may not be present in the log or the recorded trace.…”

“…BigDebug [16] is a debugger for Apache Spark [10] which introduces the concept of simulated breakpoint that does not stop the execution nor freezes the system waiting for the resolution of the breakpoint. Instead, it stores the information necessary to 3:22 replay the environment and then continues the execution.…”