Garbage collection of timestamped data in Stampede

Abstract

Stampede is a parallel programming system to facilitate the programming of interactive m ultimedia applications on clusters of SMPs.In a Stampede application, a variable number of threads can communicate data items to each other via channels, which are distributed, synchronized data structures containing timestamped data such as images from a video camera. Channels are not queue-like: threads may produce and consume items out of timestamp order; they may produce and consume items sparsely skipping timestamps, and multiple threads including newly created threads may consume an item in a channel. These exibilities are required due to the complex dynamic parallel structure of applications, to support increased parallelism, and because of real-time requirements.Under these circumstances, a key issue is the garbage collection condition": When can an item in a channel be garbage collected?In this paper we specify precisely Stampede's semantics concerning timestamps, and we describe two associated garbage collection conditions a weak condition, and a more expensive but stronger condition. We then describe a distributed, concurrent algorithm that implements these two GC conditions. We present performance numbers that show little or no application-level performance penalty to using this algorithm for aiding automatic garbage collection in a cluster. We conclude with some remarks about the implementation in the Stampede system.

A Comparative Study of Stampede Garbage Collection Algorithms

“…If nothing is known about the application task graph, then similar to PDES, there is a necessity in Stampede to compute GVT to enable garbage collection. The less restrictive nature of the Stampede programming model allows conception of different types of algorithms for GVT calculation like the one described in [8]. In [5] we have proposed yet another algorithm that uses application-level knowledge enabling garbage collection based entirely on local events with no reliance on any global mechanism.…”

“…There are three competing mechanisms for GC in Stampede: a REFerence count based garbage collector (REF), a Transparent Garbage Collector (TGC) [8], and a Dead timestamp based Garbage Collector (DGC) [5]. Each algorithm represents a specific point in the tradeoff between information needed to be provided to the runtime system and the corresponding aggressiveness for eliminating garbage.…”

“…The TGC algorithm [8] maintains two state variables on behalf of each thread. The first is the thread virtual time, and the second is the thread keep time.…”

“…In a companion paper [8], we have presented a distributed garbage collection algorithm for Stampede that does not use any application-level knowledge. In a more recent paper [5], we have introduced a new algorithm, called dead-timestamp based garbage collection (DGC), that uses an application-level task graph to make runtime decisions on the interest set for timestamps.…”

Adaptive Resource Utilization via Feedback Control for Streaming Applications

“…Efficient implementation of the application will attempt to minimize resource utilization wasted in maintaining such data. Stampede run-time system contains garbage collection (GC) algorithms [18,6,14] based on timestamp visibility that free data elements as soon as the run-time system can be certain that they are not going to be used by the application. These Stampede GC algorithms differ from traditional GC in their logic governing garbage collection in that they assert data will not be used in the future.…”