Automated optimization of decoder hyper-parameters for online LVCSR

Chandrashekaran, Akshay; Lane, Ian

doi:10.1109/slt.2016.7846303

Cited by 5 publications

(3 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As a result, optimization using both algorithm performance and hardware cost should be considered, especially for edge devices. Hardware-related costs can be measured in different ways; e.g., through energy consumption (Hernández-Lobato et al 2016) or memory utilization (Chandra and Lane 2016). In many cases, these measures are estimated as a function of the hyperparameters.…”

Section: Multi-objective Hpo: Typical Objectivesmentioning

confidence: 99%

A survey on multi-objective hyperparameter optimization algorithms for machine learning

2022

View full text Add to dashboard Cite

Hyperparameter optimization (HPO) is a necessary step to ensure the best possible performance of Machine Learning (ML) algorithms. Several methods have been developed to perform HPO; most of these are focused on optimizing one performance measure (usually an error-based measure), and the literature on such single-objective HPO problems is vast. Recently, though, algorithms have appeared that focus on optimizing multiple conflicting objectives simultaneously. This article presents a systematic survey of the literature published between 2014 and 2020 on multi-objective HPO algorithms, distinguishing between metaheuristic-based algorithms, metamodel-based algorithms and approaches using a mixture of both. We also discuss the quality metrics used to compare multi-objective HPO procedures and present future research directions.

show abstract

Section: Multi-objective Hpo: Typical Objectivesmentioning

confidence: 99%

A survey on multi-objective hyperparameter optimization algorithms for machine learning

2022

View full text Add to dashboard Cite

show abstract

“…However, it can be inferred that this comparison is not made for an equal budget of candidate solution evaluations and is therefore difficult to generalize from. Chandrashekaran et al [16] report the number of candidate evaluations in a comparative optimization of a speech recognition model. However, the use of a very small population size again limits the generalizability of the conclusions.…”

Section: Applications Of Bayesian Optimizationmentioning

confidence: 99%

Investigating Bayesian Optimization for rail network optimization

Hickish

Fletcher

Harrison

2019

International Journal of Rail Transportation

View full text Add to dashboard Cite

Optimizing the operation of rail networks using simulations is an ongoing task where heuristic methods such as Genetic Algorithms have been applied. However, these simulations are often expensive to compute and consequently, because the optimization methods require many (typically >10 4 ) repeat simulations, the computational cost of optimization is dominated by them. This paper examines Bayesian Optimization and benchmarks it against the Genetic Algorithm method. By applying both methods to test-tasks seeking to maximize passenger satisfaction by optimum resource allocation, it is experimentally determined that a Bayesian Optimization implementation finds 'good' solutions in an order of magnitude fewer simulations than a Genetic Algorithm. Similar improvement for realworld problems will allow the predictive power of detailed simulation models to be used for a wider range of network optimization tasks. To the best of the authors' knowledge, this paper documents the first application of Bayesian Optimization within the field of rail network optimization.

show abstract

“…For the decoder-CBO experiments (CBO-D), we used the manually selected model hyper-parameter configuration and ran 20 iterations of constrained Bayesian Optimization. The choice of 20 rounds of decoder hyper-parameter optimization is based on the methodology tried in [18].…”

Section: Optimization Pipelinementioning

confidence: 99%

Hierarchical Constrained Bayesian Optimization for Feature, Acoustic Model and Decoder Parameter Optimization

Chandrashekaran

Lane

2017

Interspeech 2017

Self Cite

View full text Add to dashboard Cite

We describe the implementation of a hierarchical constrained Bayesian Optimization algorithm and it's application to joint optimization of features, acoustic model structure and decoding parameters for deep neural network (DNN)-based large vocabulary continuous speech recognition (LVCSR) systems. Within our hierarchical optimization method we perform constrained Bayesian optimization jointly of feature hyper-parameters and acoustic model structure in the first-level, and then perform an iteration of constrained Bayesian optimization for the decoder hyper-parameters in the second. We show the the proposed hierarchical optimization method can generate a model with higher performance than a manually optimized system on a server platform. Furthermore, we demonstrate that the proposed framework can be used to automatically build real-time speech recognition systems for graphics processing unit (GPU)-enabled embedded platforms that retain similar accuracy to a server platform, while running with constrained computing resources.

show abstract

Automated optimization of decoder hyper-parameters for online LVCSR

Cited by 5 publications

References 9 publications

A survey on multi-objective hyperparameter optimization algorithms for machine learning

A survey on multi-objective hyperparameter optimization algorithms for machine learning

Investigating Bayesian Optimization for rail network optimization

Hierarchical Constrained Bayesian Optimization for Feature, Acoustic Model and Decoder Parameter Optimization

Contact Info

Product

Resources

About