This paper introduces a new open source platform for end-toend speech processing named ESPnet. ESPnet mainly focuses on end-to-end automatic speech recognition (ASR), and adopts widely-used dynamic neural network toolkits, Chainer and Py-Torch, as a main deep learning engine. ESPnet also follows the Kaldi ASR toolkit style for data processing, feature extraction/format, and recipes to provide a complete setup for speech recognition and other speech processing experiments. This paper explains a major architecture of this software platform, several important functionalities, which differentiate ESPnet from other open source ASR toolkits, and experimental results with major ASR benchmarks.

No abstract

Sequence-to-sequence (seq2seq) approach for low-resource ASR is a relatively new direction in speech research. The approach benefits by performing model training without using lexicon and alignments. However, this poses a new problem of requiring more data compared to conventional DNN-HMM systems. In this work, we attempt to use data from 10 BABEL languages to build a multilingual seq2seq model as a prior model, and then port them towards 4 other BABEL languages using transfer learning approach. We also explore different architectures for improving the prior multilingual seq2seq model. The paper also discusses the effect of integrating a recurrent neural network language model (RNNLM) with a seq2seq model during decoding. Experimental results show that the transfer learning approach from the multilingual model shows substantial gains over monolingual models across all 4 BABEL languages. Incorporating an RNNLM also brings significant improvements in terms of %WER, and achieves recognition performance comparable to the models trained with twice more training data.

On 2019 August 14, the LIGO and Virgo Collaborations detected gravitational waves from a black hole and a 2.6 solar mass compact object, possibly the first neutron star-black hole merger. In search of an optical counterpart, the Dark Energy Survey (DES) obtained deep imaging of the entire 90% confidence level localization area with Blanco/DECam 0, 1, 2, 3, 6, and 16 nights after the merger. Objects with varying brightness were detected by the DES Pipeline, and we systematically reduced the candidate counterparts through catalog matching, light-curve properties, host-galaxy photometric redshifts, Southern Astrophysical Research spectroscopic follow-up observations, and machine-learning-based photometric classification. All candidates were rejected as counterparts to the merger. To quantify the sensitivity of our search, we applied our selection criteria to full light-curve simulations of supernovae and kilonovae as they would appear in the DECam observations. Because the source class of the merger was uncertain, we utilized an agnostic, three-component kilonova model based on tidally disrupted neutron star (NS) ejecta properties to quantify our detection efficiency of a counterpart if the merger included an NS. We find that, if a kilonova occurred during this merger, configurations where the ejected matter is greater than 0.07 solar masses, has lanthanide abundance less than 10 −8.56 , and has a velocity between 0.18c and 0.21c are disfavored at the 2σ level. Furthermore, we estimate that our background reduction methods are capable of associating gravitational wave signals with a detected electromagnetic counterpart at the 4σ level in 95% of future follow-up observations.