Physics-informed neural networks (PINNs) [1] have recently arisen as a promising solution methodology for inverse problems. The solution is approximated with a neural network trained by minimizing the residual of a partial differential equation. This work aims to pinpoint the strengths and weaknesses of PINNs in relation to the classical adjoint optimization. We present an incremental comparison of PINNs w.r.t. the classical adjoint optimization in the context of inverse problems. To this end, we consider the three key ingredients (a) the forward solver, (b) the Ansatz space of the optimization variable, and (c) the sensitivity computation. The empirical investigation is performed for full waveform inversion, where the unknown is a scaling function of the density field to locate internal voids [2]. PINN-based approaches, as presented in [3], represent both the solution and the scaling function with separate neural networks and perform a nested minimization of the emerging residuals.