Understanding the Mechanics of SPIGOT: Surrogate Gradients for Latent Structure Learning

Abstract

Latent structure models are a powerful tool for modeling language data: they can mitigate the error propagation and annotation bottleneck in pipeline systems, while simultaneously uncovering linguistic insights about the data. One challenge with end-to-end training of these models is the argmax operation, which has null gradient. In this paper, we focus on surrogate gradients, a popular strategy to deal with this problem. We explore latent structure learning through the angle of pulling back the downstream learning objective. In this paradigm, we discover a principled motivation for both the straight-through estimator (STE) as well as the recently-proposed SPIGOT-a variant of STE for structured models. Our perspective leads to new algorithms in the same family. We empirically compare the known and the novel pulled-back estimators against the popular alternatives, yielding new insight for practitioners and revealing intriguing failure cases.

Learning with Latent Structures in Natural Language Processing: A Survey

“…Effectively, this procedure performs a one-step gradient descent to induce an approximation of the optimal structure. Mihaylova et al (2020) noted that this formulation recovers the identity STE in Equation (1):…”

“…Being mindful of the structural constraints for STE can help improve the learning process. To see this, we first examine an alternative formulation of STE as proposed by Mihaylova et al (2020). Consider the hypothetical case where the optimal structure z * is accessible.…”

“…Inspired by SPIGOT, Mihaylova et al (2020) explored more numbers of (projected) gradient update steps, an alternative loss based on the CRF loss (Lafferty et al, 2001) in the place of the structured perceptron loss, and exponentiated gradient updates (Kivinen and Warmuth, 1997). Nevertheless, they did not observe significant improvement in the structured case.…”

SynJax: Structured Probability Distributions for JAX

“…where T is the set of all possible structures and Z is a normalization often called partition function. This equation can be thought of as a softmax equivalent over an extremely large set of structured outputs that share sub-structures (Sutton and Mc-Callum, 2007;Mihaylova et al, 2020).…”