Two Birds with One Stone: Investigating Invertible Neural Networks for Inverse Problems in Morphology

Abstract

Most problems in natural language processing can be approximated as inverse problems such as analysis and generation at variety of levels from morphological (e.g., cat+Plural↔cats) to semantic (e.g., (call + 1 2)↔“Calculate one plus two.”). Although the tasks in both directions are closely related, general approach in the field has been to design separate models specific for each task. However, having one shared model for both tasks, would help the researchers exploit the common knowledge among these problems with reduced time and memory requirements. We investigate a specific class of neural networks, called Invertible Neural Networks (INNs) (Ardizzone et al. 2019) that enable simultaneous optimization in both directions, hence allow addressing of inverse problems via a single model. In this study, we investigate INNs on morphological problems casted as inverse problems. We apply INNs to various morphological tasks with varying ambiguity and show that they provide competitive performance in both directions. We show that they are able to recover the morphological input parameters, i.e., predicting the lemma (e.g., cat) or the morphological tags (e.g., Plural) when run in the reverse direction, without any significant performance drop in the forward direction, i.e., predicting the surface form (e.g., cats).

Robust Reconstruction of the Void Fraction from Noisy Magnetic Flux Density Using Invertible Neural Networks

“…Even though there have been recent attempts to use INNs as surrogate models for solving inverse problems, such as [ 52 ] for inverse problems in physical systems governed by Partial Differential Equations (PDEs), Ref. [ 53 ] for inverse problem in morphology, Ref. [ 54 ] for inverse problem in medical imaging, or [ 55 ] for inverse design of optical lenses.…”

Bayesian geophysical inversion using invertible neural networks

“…INNs provide bijective mappings between inputs (models) and outputs (data), and can be trained to estimate posterior pdfs by introducing additional latent variables in the outputs (data) side. They have been used to solve inverse problems in medicine (Ardizzone et al 2018), astrophysics (Osborne et al 2019), optical imaging (Adler et al 2019;Moran et al 2018) and morphology (Sahin & Gurevych 2020). In this study we use INNs to solve seismic tomographic inverse problems.…”

Bayesian Geophysical Inversion Using Invertible Neural Networks

“…INNs provide bijective (two‐way) mappings between inputs (models) and outputs (data), and can be trained to estimate posterior pdfs by introducing additional latent variables in the outputs (data) side. They have been used to solve inverse problems in medicine (Ardizzone et al., 2018), astrophysics (Osborne et al., 2019), optical imaging (Adler et al., 2019; Moran et al., 2018) and morphology (Sahin & Gurevych, 2020). INNs have also been used to solve a variational problem to parameterize uncertainty for reservoir characterization (Rizzuti et al., 2020), and the idea of auxiliary variables has also been used in seismic full waveform inversion (Huang et al., 2018).…”