Exact optimal values of step-size coefficients for boundedness of linear multistep methods

Lóczi, Lajos

doi:10.1007/s11075-017-0354-5

Cited by 2 publications

(2 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This offers a more flexible way to control the model behavior. Second, DLCL has an arbitrary size of the past history window, while LMM generally takes a limited history into account (Lóczi, 2018). Also, recent work shows successful applications of LMM in computer vision, but only two previous steps are used in their LMM-like system (Lu et al, 2018).…”

Section: Dynamic Linear Combination Of Layersmentioning

confidence: 99%

Learning Deep Transformer Models for Machine Translation

Wang¹,

Li²,

Xiao³

et al. 2019

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics

461

214

View full text Add to dashboard Cite

Transformer is the state-of-the-art model in recent machine translation evaluations. Two strands of research are promising to improve models of this kind: the first uses wide networks (a.k.a. Transformer-Big) and has been the de facto standard for the development of the Transformer system, and the other uses deeper language representation but faces the difficulty arising from learning deep networks. Here, we continue the line of research on the latter. We claim that a truly deep Transformer model can surpass the Transformer-Big counterpart by 1) proper use of layer normalization and 2) a novel way of passing the combination of previous layers to the next. On WMT'16 English-German, NIST OpenMT'12 Chinese-English and larger WMT'18 Chinese-English tasks, our deep system (30/25-layer encoder) outperforms the shallow Transformer-Big/Base baseline (6-layer encoder) by 0.4∼2.4 BLEU points. As another bonus, the deep model is 1.6X smaller in size and 3X faster in training than Transformer-Big 1 . * Corresponding author. 1 The source code is available at https://github. com/wangqiangneu/dlcl

show abstract

Section: Dynamic Linear Combination Of Layersmentioning

confidence: 99%

Learning Deep Transformer Models for Machine Translation

Wang¹,

Li²,

Xiao³

et al. 2019

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics

461

214

View full text Add to dashboard Cite

show abstract

“…These coefficients govern the largest allowable step-size guaranteeing certain monotonicity or boundedness properties of the LMM, including the TVD and SSP properties [15]. These properties are relevant, for example, in the time integration of method-of-lines semi-discretizations of hyperbolic conservation laws [41,19,35].…”

Section: Motivation and Main Resultsmentioning

confidence: 99%

Optimal subsets in the stability regions of multistep methods

Lóczi

2019

Numer Algor

Self Cite

View full text Add to dashboard Cite

In this work we study the stability regions of linear multistep or multiderivative multistep methods for initial-value problems by using techniques that are straightforward to implement in modern computer algebra systems. In many applications, one is interested in (i ) checking whether a given subset of the complex plane (e.g. a sector, disk, or parabola) is included in the stability region of the numerical method, (ii ) finding the largest subset of a certain shape contained in the stability region of a given method, or (iii ) finding the numerical method in a parametric family of multistep methods whose stability region contains the largest subset of a given shape.First we describe a simple procedure to exactly calculate the stability angle α in the definition of A(α)-stability by representing the root locus curve (RLC) of the multistep method as an implicit algebraic curve. As an illustration, we consider two finite families of implicit multistep methods. We exactly compute the stability angles for the k-step BDF methods (3 ≤ k ≤ 6) and discover that the values of tan(α) are surprisingly simple algebraic numbers of degree 2, 2, 4 and 2, respectively. In contrast, the corresponding values of tan(α) for the k-step second-derivative multistep methods of Enright (3 ≤ k ≤ 7) are much more complicated; the smallest algebraic degree here is 22.Next we determine the exact value of the stability radius in the BDF family for each 3 ≤ k ≤ 6, that is, the radius of the largest disk in the left half of the complex plane, symmetric with respect to the real axis, touching the imaginary axis and lying in the stability region of the corresponding method. These radii turn out to be algebraic numbers of degree 2, 3, 5 and 5, respectively.Finally, we demonstrate how some Schur-Cohn-type theorems of recursive nature and not relying on the RLC method can be used to exactly solve some optimization problems within infinite parametric families of multistep methods. As an example, we choose a two-parameter family of implicit-explicit (IMEX) methods: we identify the unique method having the largest stability angle in the family, then we find the unique method in the same family whose stability region contains the largest parabola. * LLoczi@cs.elte.hu,

show abstract

Exact optimal values of step-size coefficients for boundedness of linear multistep methods

Cited by 2 publications

References 15 publications

Learning Deep Transformer Models for Machine Translation

Learning Deep Transformer Models for Machine Translation

Optimal subsets in the stability regions of multistep methods

Contact Info

Product

Resources

About