Ofc

Abstract

Cloud applications based on the "Functions as a Service" (FaaS) paradigm have become very popular. Yet, due to their stateless nature, they must frequently interact with an external data store, which limits their performance. To mitigate this issue, we introduce OFC, a transparent, vertically and horizontally elastic in-memory caching system for FaaS platforms, distributed over the worker nodes. OFC provides these benefits cost-effectively by exploiting two common sources of resource waste: (i) most cloud tenants overprovision the memory resources reserved for their functions because their footprint is non-trivially input-dependent and (ii) FaaS providers keep function sandboxes alive for several minutes to avoid cold starts. Using machine learning models adjusted for typical function input data categories (e.g., multimedia formats), OFC estimates the actual memory resources required by each function invocation and hoards the remaining capacity to feed the cache. We build our OFC prototype based on enhancements to the OpenWhisk FaaS platform, the Swift persistent object store, and the RAM-Cloud in-memory store. Using a diverse set of workloads, we show that OFC improves by up to 82 % and 60 % respectively the execution time of single-stage and pipelined functions. CCS Concepts: • Computer systems organization → Cloud computing; • Software and its engineering → n-tier architectures. * A part of work done while Jinho Hwang was at IBM Research.

Āpta: Fault-tolerant object-granular CXL disaggregated memory for accelerating FaaS

“…The compute server OS sets the translation table base address register -ZMMU [26] or extended page table pointer [45]. 3 Other object cache allocation policies can be employed -e.g., fraction of LLC capacity per CPU, dynamically based on predictors of object hotness or reuse potential [64] etc. The exploration of allocation policies is orthogonal.…”

“…Āpta addresses a chief inefficiency of FaaS -data transfer overheads and provides a fault-tolerant DM system for FaaS applications. Reducing communication overheads in FaaS: Several works [55], [64], [77], [81], [83], [88] aim to provide softwarebased caches at compute servers to cache objects. These works reinforce the potential of caching to improve performance despite being connected by fast networks.…”

“…Āpta's method for transforming the non fault-tolerant coherence protocol into a highly-available one can easily be applied to upcoming versions of CXL. stores S3 [7] No No High Low Pond [50], Kona [8] Yes (cacheline, write-back, No, N/A N/A) Yes Low Low ThymesisFlow [74] LegoOS [79] Yes (page, write-back, No, N/A, N/A) Yes Low Medium Clio-KV [31] Yes (object, write-through, Yes, No, N/A) Yes Low Medium MIND [48] Yes (page, write-back, Yes, MSI, sync) Yes Low Medium OFC [64] Yes ( Contributions.…”

Floki

“…Works like Crucial [2], Cloudburst [19], OFC [14], SONIC [12], and SAND [1] focus on inter-function data sharing, disaggregation and physical co-location, or data locality exploitation. However, they need static provisioning of per-node cache resources or focus on small data sizes.…”

Building efficient and practical machine learning systems

“…In addition to the excellent interpretability and accuracy, DTs have extremely low computation overhead and inference latency. Consequently, they are applicable to many scenarios with strict latency and resource constraints [247,310]. Besides, DTs also exhibit other benefits, including robust performance under dynamic system environments, requiring less training data and no data preprocessing overhead during inference.…”