Real-Time Machine Learning

Nishihara, Robert; Moritz, Philipp; Wang, Stephanie; Tumanov, Alexey; Paul, William; Liaw, Richard; Niknami, Mehrdad; Jordan, Michael I.; Stoica, Ion

doi:10.1145/3102980.3102998

Cited by 41 publications

(24 citation statements)

References 13 publications

(8 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Efficiently processing stream data is a key part in the framework as the tracking process need to be detached from the visual model via a robust communication interface without additional delays. Redis databases are also starting to play an important role in distributed learning an deployment of model free Reinforcement Learning (RL) [25] in several studies [26,27]. Due to the computational load and iterations required for deep reinforcement learning based control approaches, Redis lends itself very well for sharing distributed data through asynchronous controller updates [28].…”

Section: Distributed Frame and Data Streamingmentioning

confidence: 99%

Parallel Real-Time Tracking and 3D Reconstruction with TBB for Intelligent Control and Smart Sensing Framework

Mkhoyan

Visser

Breuker

2020

AIAA Scitech 2020 Forum

View full text Add to dashboard Cite

Recent advancements in aircraft controllers paired with increasingly flexible aircraft designs create the need for adaptive and intelligent control systems. To correctly capture the motion of a flexible aircraft wing and provide feedback to the controller, a large number of states (nodes along the span) must be monitored in real-time. Visual sensing methods carry the promise of flexibility needed for this type of smart sensing and control. However, visual sensing requires capturing and tracking keypoint features (marker tracking), while detecting thereof from a feature-rich image can be a computationally intensive task. The computational effort significantly increases with image size or when an image stereo pair is used to find matching keypoints. In this study, a parallel approach is presented with Threading Building Blocks (TBB), using sub-matrix computations, for extraction of corresponding keypoints from an image-stereo pair, and triangulation with the Direct Linear Transform (DLT) method to reconstruct the 3D position of the object in space. Additional robustness is investigated by implementing a Kalman filter for tracking prediction during the domain transition between the sub-matrices. Furthermore, a flexible simulation framework is set up for smart sensing with a coupled unsteady aeroservoelastic model of a 3D wing and a visual model to test the method for intelligent control feedback in a simulation environment. The methodology is tested in a laboratory environment with a stereo camera setup, and in a virtual environment, where the virtual camera parameters are reconstructed to meet a stereo setup. The proposed approach aims to advance the state-of-the-art in smart sensing, particularly in the context of real-time state estimation of aeroelastic structures and control feedback. The parallel approach shows a significant improvement of speed and efficiency, allowing real-time computation from a live image stream at 50 fps.

show abstract

Section: Distributed Frame and Data Streamingmentioning

confidence: 99%

Parallel Real-Time Tracking and 3D Reconstruction with TBB for Intelligent Control and Smart Sensing Framework

Mkhoyan

Visser

Breuker

2020

AIAA Scitech 2020 Forum

View full text Add to dashboard Cite

show abstract

“…For example, CIEL represents a program as an unstructured "dynamic task graph" in which tasks can tail-recursively spawn other tasks, and imperative control-flow constructs are transformed into continuation-passing style [29]. Nishihara et al recently described a system for "real-time machine learning" that builds on these ideas, and adds a decentralized and hierarchical scheduler to improve the latency of task dispatch [30]. Programming models based on dynamic task graphs are a direct fit for algorithms that make recursive traversals over dynamic data structures, such as parse trees [15].…”

Section: Related Workmentioning

confidence: 99%

Dynamic control flow in large-scale machine learning

Yuan

Abadi

Barham

et al. 2018

Proceedings of the Thirteenth EuroSys Conference

View full text Add to dashboard Cite

Many recent machine learning models rely on fine-grained dynamic control flow for training and inference. In particular, models based on recurrent neural networks and on reinforcement learning depend on recurrence relations, data-dependent conditional execution, and other features that call for dynamic control flow. These applications benefit from the ability to make rapid control-flow decisions across a set of computing devices in a distributed system. For performance, scalability, and expressiveness, a machine learning system must support dynamic control flow in distributed and heterogeneous environments. This paper presents a programming model for distributed machine learning that supports dynamic control flow. We describe the design of the programming model, and its implementation in TensorFlow, a distributed machine learning system. Our approach extends the use of dataflow graphs to represent machine learning models, offering several distinctive features. First, the branches of conditionals and bodies of loops can be partitioned across many machines to run on a set of heterogeneous devices, including CPUs, GPUs, and custom ASICs. Second, programs written in our model support automatic differentiation and distributed gradient computations, which are necessary for training machine learning models * Work done primarily at Google Brain. that use control flow. Third, our choice of non-strict semantics enables multiple loop iterations to execute in parallel across machines, and to overlap compute and I/O operations.We have done our work in the context of TensorFlow, and it has been used extensively in research and production. We evaluate it using several real-world applications, and demonstrate its performance and scalability.

show abstract

“…Ray [29] optimizes this centralized design by enabling a worker to spawn and execute some tasks locally. If a task accesses data on other workers, however, it must still perform synchronous operations with a controller.…”

Section: Cloud Framework Control Planesmentioning

confidence: 99%

“…This tradeoff introduces a fundamental limit on how much these systems can parallelize a job. Increasing parallelization has many benefits, including improved fault tolerance, load balancing, and faster performance [17,27,29,30,43]. In practice, however, increasing parallelism past even a few hundred cores quickly hits the control plane's scaling limit, making jobs run slower [24,32].…”

Section: Introductionmentioning

confidence: 99%

Decoupling the control plane from program control flow for flexibility and performance in cloud computing

Mashayekhi

Shah

et al. 2018

Proceedings of the Thirteenth EuroSys Conference

View full text Add to dashboard Cite

Existing cloud computing control planes do not scale to more than a few hundred cores, while frameworks without control planes scale but take seconds to reschedule a job. We propose an asynchronous control plane for cloud computing systems, in which a central controller can dynamically reschedule jobs but worker nodes never block on communication with the controller. By decoupling control plane traffic from program control flow in this way, an asynchronous control plane can scale to run millions of computations per second while being able to reschedule computations within milliseconds. We show that an asynchronous control plane can match the scalability and performance of TensorFlow and MPI-based programs while rescheduling individual tasks in milliseconds. Scheduling an individual task takes 1μs, such that a 1,152 core cluster can schedule over 120 million tasks/second and this scales linearly with the number of cores. The ability to schedule huge numbers of tasks allows jobs to be divided into very large numbers of tiny tasks, whose improved load balancing can speed up computations 2.1-2.3×. CCS CONCEPTS • Computer systems organization → Cloud computing; • Computing methodologies → Parallel algorithms;

show abstract

Real-Time Machine Learning

Cited by 41 publications

References 13 publications

Parallel Real-Time Tracking and 3D Reconstruction with TBB for Intelligent Control and Smart Sensing Framework

Parallel Real-Time Tracking and 3D Reconstruction with TBB for Intelligent Control and Smart Sensing Framework

Dynamic control flow in large-scale machine learning

Decoupling the control plane from program control flow for flexibility and performance in cloud computing

Contact Info

Product

Resources

About