Recommender System for Optimal Distributed Deep Learning in Cloud Datacenters

Abstract

With the modern advancements in Deep Learning architectures, and abundant research consistently being put forward in areas such as computer vision, natural language processing and forecasting. Models are becoming complicated and datasets are growing exponentially in size demanding high performing and faster computing machines from researchers and engineers. TensorFlow provides a wide range of distributed deep learning high-level APIs to address this issue, that can scale deep learning training from one machine to more than one. In this paper, we have investigated the performance of computing clusters utilizing those APIs. We created clusters of different sizes and discuss performance issues of distributed deep learning under high latency and poor communication conditions. To address the challenge of finding the optimal cluster for fast distributed deep learning, we have proposed a recommendation system, that can provide an optimal cluster size for fastest training time, given batch size and networking latency. Our results show that using a 2 machine cluster is both faster and cheaper than a four machine cluster for certain algorithms when network delay is high.

Edge AI: A survey

“…It might be everything from a storefront to a factory to a hospital to the ubiquitous traffic signals, driverless equipment, and mobile phones. Increased automation is a goal of businesses across all sectors since it leads to greater efficiency, productivity, and security [26]. Computer programmes may assist with this by learning to spot patterns and reliably carry out the same actions over and over [27].…”

“…Networking: Standardizing the networking infrastructure can help ensure that different components can communicate with each other seamlessly. This includes standardizing protocols, cabling, and network topology [26]. Management and Monitoring: Standardizing the management and monitoring tools can help simplify deployment and reduce the risk of human error.…”

“…Similarly, generic processing functionalities would aid developers to mix and match when devising edge solutions [24]. New offerings such as SDN and NFV could be used to achieve dynamic configurations for edge processing [26]. In this section, we focus on open research challenges that are each important for the broad implementation of Edge Computing [147,155].…”

AI for Next Generation Computing: Emerging Trends and Future Directions

“…Among them, edge offloading is a distributed computing paradigm that provides computing services for edge caching, edge training, and edge inference. By integrating methods such as Distributed Machine Learning (DML), Deep Reinforcement Learning (DRL) and Collaborative Machine Learning (CML) into the edge computing, it is beneficial to cope with the explosive growth of communication and computing of emerging IoT applications [187], and achieve the energy-efficient and real-time processing [188].…”