ON-TRAC Consortium Systems for the IWSLT 2022 Dialect and Low-resource Speech Translation Tasks

Speeding up the Training of Neural Networks with the One-Step Procedure

“…Heuristically, in the simplest statistical experiments (i.i.d. or generalized linear models with controlled covariates variables as in [18]), if the initial guess estimator ϑ n is n δ 2consistent (with 1 2 < δ ≤ 1), i.e. for any > 0 there exists a constant C such that…”

“…Snubbed during the winter of artificial intelligence, the fast increase in computing capacity of the last decade allowed to exploit their potential, leading to their supremacy when considering machine learning. Indeed, they were successfully operated to solve a wide range of problems: image recognition [26], machine translation and automatic speech recognition [1], etc. However, these networks became deeper and deeper and increasingly complex.…”

“…2.2 for the definition) and the Fisher information matrix uniformly continuous. But it had been recently shown [13,14] that for a n δ 2 -consistent initial sequence of guess estimators (with 1 2 < δ ≤ 1) and a Lipschitz Fisher information matrix, the sequence of One-Step estimators is still consistent, asymptotically normal and efficient. In this setting, for a initial sequence which is neither asymptotically rate nor variance efficient, the new sequence is asymptotically rate and variance efficient.…”