A Scalable and Cloud-Native Hyperparameter Tuning System

George, Johnu; Gao, Ce; Liu, Richard; Liu, Hou Gang; Tang, Yuan Yan; Pydipaty, Ramdoot; Saha, Amit K.

doi:10.48550/arxiv.2006.02085

Cited by 4 publications

(2 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…With this solution, Kubernetes manages the underlying compute resource pool and is able to efficiently schedule compute jobs. Within KubeFlow, we leverage Katib (George et al, 2020) -KubeFlow's "AutoML" framework -to efficiently explore the hyperparameter space and specify individual sweeps. As KubeFlow is an industry-grade tool, many cloud providers offer KubeFlow as a service or provide supported pathways for deploying a KubeFlow cluster, facilitating replication and compute resource scaling.…”

Section: Statement Of Needmentioning

confidence: 99%

High-performance neural population dynamics modeling enabled by scalable computational infrastructure

Patel¹,

Sedler²,

Huang³

et al. 2023

JOSS

View full text Add to dashboard Cite

Advances in neural interface technology are facilitating parallel, high-dimensional time series measurements of the brain in action. A powerful strategy for analyzing these measurements is to apply unsupervised learning techniques to uncover lower-dimensional latent dynamics that explain much of the variance in the high-dimensional measurements (Cunningham & Yu, 2014;Golub et al., 2018;Vyas et al., 2020). Latent factor analysis via dynamical systems (LFADS) (Pandarinath et al., 2018) provides a deep learning approach for extracting estimates of these latent dynamics from neural population data. The recently developed AutoLFADS framework (Keshtkaran et al., 2022) extends LFADS by using Population Based Training (PBT) (Jaderberg et al., 2017) to effectively and scalably tune model hyperparameters, a critical step for accurate modeling of neural population data. As hyperparameter sweeps are one of the most computationally demanding processes in model development, these workflows should be deployed in a computationally efficient and cost effective manner given the compute resources available (e.g., local, institutionally-supported, or commercial computing clusters). The initial implementation of AutoLFADS used the Ray library (Moritz et al., 2018) to enable support for specific local and commercial cloud workflows. We extend this support, by providing additional options for training AutoLFADS models using local clusters in a container-native approach (e.g., Docker, Podman), unmanaged compute clusters leveraging Ray, and managed compute clusters leveraging KubeFlow and Kubernetes orchestration.

show abstract

Section: Statement Of Needmentioning

confidence: 99%

High-performance neural population dynamics modeling enabled by scalable computational infrastructure

Patel¹,

Sedler²,

Huang³

et al. 2023

JOSS

View full text Add to dashboard Cite

show abstract

“…Kubeflow functionality echoes that of Kubernetes, as it aims to deploy, scale, and manage ML workloads [65]. Kubeflow also allows running automated ML tasks and supports hyperparameter tuning [66], thus supporting end-to-end ML workflows [67].…”

Section: Workflow Management Systemsmentioning

confidence: 99%

Autonomy and Intelligence in the Computing Continuum: Challenges, Enablers, and Future Directions for Orchestration

Kokkonen¹,

Lovén²,

Motlagh³

et al. 2022

Preprint

View full text Add to dashboard Cite

Future AI applications require performance, reliability, and privacy that the existing, cloud-dependant system architectures cannot provide. In this article, we study orchestration in the device-edge-cloud continuum, and focus on AI for edge, that is, the AI methods used in resource orchestration. We claim that to support the constantly growing requirements of intelligent applications in the device-edge-cloud computing continuum, resource orchestration needs to embrace edge AI and emphasize local autonomy and intelligence. To justify the claim, we provide a general definition for continuum orchestration, and look at how current and emerging orchestration paradigms are suitable for the computing continuum. We describe certain major emerging research themes that may affect future orchestration, and provide an early vision of an orchestration paradigm that embraces those research themes. Finally, we survey current key edge AI methods and look at how they may contribute into fulfilling the vision of future continuum orchestration.

show abstract

Massively Parallel Genetic Optimization Through Asynchronous Propagation of Populations

Taubert

Weiel

Coquelin

et al. 2023

Lecture Notes in Computer Science

View full text Add to dashboard Cite

We present , an evolutionary optimization algorithm and software package for global optimization and in particular hyperparameter search. For efficient use of HPC resources, omits the synchronization after each generation as done in conventional genetic algorithms. Instead, it steers the search with the complete population present at time of breeding new individuals. We provide an MPI-based implementation of our algorithm, which features variants of selection, mutation, crossover, and migration and is easy to extend with custom functionality. We compare to the established optimization tool . We find that is up to three orders of magnitude faster without sacrificing solution accuracy, demonstrating the efficiency and efficacy of our lazy synchronization approach. Code and documentation are available at https://github.com/Helmholtz-AI-Energy/propulate/.

show abstract

A Scalable and Cloud-Native Hyperparameter Tuning System

Cited by 4 publications

References 0 publications

High-performance neural population dynamics modeling enabled by scalable computational infrastructure

High-performance neural population dynamics modeling enabled by scalable computational infrastructure

Autonomy and Intelligence in the Computing Continuum: Challenges, Enablers, and Future Directions for Orchestration

Massively Parallel Genetic Optimization Through Asynchronous Propagation of Populations

Contact Info

Product

Resources

About