Split Learning: A Resource Efficient Model and Data Parallel Approach for Distributed Deep Learning

Vepakomma, Praneeth; Raskar, Ramesh

doi:10.1007/978-3-030-96896-0_19

Cited by 13 publications

(15 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Collaborative learning will be applied when the multiple data owners provide data for a subset of features. There also exist other techniques such as transfer learning or split learning [65] that can support training models while preserving data privacy. We believe that in DEML the resource scheduling algorithms are the key focus to enable the training of models on datasets and distributed computing resources.…”

Section: Impact On Deml-rcce Software Componentsmentioning

confidence: 99%

Making distributed edge machine learning for resource-constrained communities and environments smarter: contexts and challenges

Truong

Truong-Huu

Cao

2022

J Reliable Intell Environ

View full text Add to dashboard Cite

The maturity of machine learning (ML) development and the decreasing deployment cost of capable edge devices have proliferated the development and deployment of edge ML solutions for critical IoT-based business applications. The combination of edge computing and ML not only addresses the development cost barrier, but also solves the obstacles due to the lack of powerful cloud data centers. However, not only the edge ML research and development is still at an early stage and requires substantial skills normally missed in resource-constrained communities, but also various infrastructure constraints w.r.t. network reliability and computing power, and business contexts from the resource-constrained environments require different considerations to make edge ML applications context aware through smart and intelligent runtime strategies. In this paper, we analyze representative real-world business scenarios for edge ML solutions and their contexts in resource-constrained communities and environments. We identify and map the key distinguished contexts of distributed edge ML and discuss the impacts of these contexts on data and software components and deployment models. Finally, we present key research areas, how we should approach them, and possible tooling for making edge machine learning solutions smarter in resource-constrained communities and environments.

show abstract

Section: Impact On Deml-rcce Software Componentsmentioning

confidence: 99%

Making distributed edge machine learning for resource-constrained communities and environments smarter: contexts and challenges

Truong

Truong-Huu

Cao

2022

J Reliable Intell Environ

View full text Add to dashboard Cite

show abstract

“…Here we have a list of recent SL research, and we list the datasets and data partitioning methods used in their experiments. (Vepakomma et al, 2018a)…”

Section: A the Overview Of Recent Sl Researchmentioning

confidence: 99%

International Comparison on the Application of Information Technology in High School Mathematics Textbooks

Zhou

2021

School Mathematics Textbooks in China

View full text Add to dashboard Cite

Ultra-dense networks are widely regarded as a promising solution to explosively growing applications of Internet-of-Things (IoT) mobile devices (IMDs). However, complicated and severe interferences need to be tackled properly in such networks. To this end, both orthogonal multiple access (OMA) and non-orthogonal multiple access (NOMA) are utilized at first. Then, in order to attain a goal of green and secure computation offloading, under the proportional allocation of computational resources and the constraints of latency and security cost, joint device association, channel selection, security service assignment, power control and computation offloading are done for minimizing the overall energy consumed by all IMDs. It is noteworthy that multi-step computation offloading is concentrated to balance the network loads and utilize computing resources fully. Since the finally formulated problem is in a nonlinear mixed-integer form, it may be very difficult to find its closed-form solution. To solve it, an improved whale optimization algorithm (IWOA) is designed. As for this algorithm, the convergence, computational complexity and parallel implementation are analyzed in detail. Simulation results show that the designed algorithm may achieve lower energy consumption than other existing algorithms under the constraints of latency and security cost.

show abstract

“…: differential privacy) [95], to federated learning (sharing model parameters rather than raw data) and learning over encrypted data [96]. Federated learning is a private distributed and decentralized machine learning method that trains the shared model locally with private data without exchanging the raw patient data [97,98]. Sheller et al demonstrated federated learning on the clinically-acquired BraTS data for institution-level segmentation tasks [99].…”

Section: • Learning Under Data Privacy Constraintmentioning

confidence: 99%

Artificial intelligence in glioma imaging: challenges and advances

et al. 2020

View full text Add to dashboard Cite

Primary brain tumors including gliomas continue to pose significant management challenges to clinicians. While the presentation, the pathology, and the clinical course of these lesions is variable, the initial investigations are usually similar. Patients who are suspected to have a brain tumor will be assessed with computed tomography (CT) and magnetic resonance imaging (MRI). The imaging findings are used by neurosurgeons to determine the feasibility of surgical resection and plan such an undertaking. Imaging studies are also an indispensable tool in tracking tumor progression or its response to treatment. As these imaging studies are non-invasive, relatively cheap and accessible to patients, there have been many efforts over the past two decades to increase the amount of clinically-relevant information that can be extracted from brain imaging. Most recently, artificial intelligence (AI) techniques have been employed to segment and characterize brain tumors, as well as to detect progression or treatment-response. However, the clinical utility of such endeavours remains limited due to challenges in data collection and annotation, model training, and in the reliability of AI-generated information.We provide a review of recent advances in addressing the above challenges. First, to overcome the challenge of data paucity, different image imputation and synthesis techniques along with annotation collection efforts are summarized. Next, various training strategies are presented to meet multiple desiderata, such as model performance, generalization ability, data privacy protection, and learning with sparse annotations. Finally, standardized performance evaluation and model interpretability methods have been reviewed. We believe that these technical approaches will facilitate the development of a fully-functional AI tool in the clinical care of patients with gliomas.

show abstract

Split Learning: A Resource Efficient Model and Data Parallel Approach for Distributed Deep Learning

Cited by 13 publications

References 6 publications

Making distributed edge machine learning for resource-constrained communities and environments smarter: contexts and challenges

Making distributed edge machine learning for resource-constrained communities and environments smarter: contexts and challenges

International Comparison on the Application of Information Technology in High School Mathematics Textbooks

Artificial intelligence in glioma imaging: challenges and advances

Contact Info

Product

Resources

About