No abstract

Multilingual neural machine translation (NMT), which translates multiple languages using a single model, is of great practical importance due to its advantages in simplifying the training process, reducing online maintenance costs, and enhancing low-resource and zero-shot translation. Given there are thousands of languages in the world and some of them are very different, it is extremely burdensome to handle them all in a single model or use a separate model for each language pair. Therefore, given a fixed resource budget, e.g., the number of models, how to determine which languages should be supported by one model is critical to multilingual NMT, which, unfortunately, has been ignored by previous work. In this work, we develop a framework that clusters languages into different groups and trains one multilingual model for each cluster. We study two methods for language clustering: (1) using prior knowledge, where we cluster languages according to language family, and (2) using language embedding, in which we represent each language by an embedding vector and cluster them in the embedding space. In particular, we obtain the embedding vectors of all the languages by training a universal neural machine translation model. Our experiments on 23 languages show that the first clustering method is simple and easy to understand but leading to suboptimal translation accuracy, while the second method sufficiently captures the relationship among languages well and improves the translation accuracy for almost all the languages over baseline methods.

Autism spectrum disorder comprises several neurodevelopmental conditions presenting symptoms in social communication and restricted, repetitive behaviors. A major roadblock for drug development for autism is the lack of robust behavioral signatures predictive of clinical efficacy. To address this issue, we further characterized, in a uniform and rigorous way, mouse models of autism that are of interest because of their construct validity and wide availability to the scientific community. We implemented a broad behavioral battery that included but was not restricted to core autism domains, with the goal of identifying robust, reliable phenotypes amenable for further testing. Here we describe comprehensive findings from two known mouse models of autism, obtained at different developmental stages, using a systematic behavioral test battery combining standard tests as well as novel, quantitative, computer-vision based systems. The first mouse model recapitulates a deletion in human chromosome 16p11.2, found in 1% of individuals with autism. The second mouse model harbors homozygous null mutations in Cntnap2, associated with autism and Pitt-Hopkins-like syndrome. Consistent with previous results, 16p11.2 heterozygous null mice, also known as Del(7Slx1b-Sept1)4Aam weighed less than wild type littermates displayed hyperactivity and no social deficits. Cntnap2 homozygous null mice were also hyperactive, froze less during testing, showed a mild gait phenotype and deficits in the three-chamber social preference test, although less robust than previously published. In the open field test with exposure to urine of an estrous female, however, the Cntnap2 null mice showed reduced vocalizations. In addition, Cntnap2 null mice performed slightly better in a cognitive procedural learning test. Although finding and replicating robust behavioral phenotypes in animal models is a challenging task, such functional readouts remain important in the development of therapeutics and we anticipate both our positive and negative findings will be utilized as a resource for the broader scientific community.

The recovery process of COVID-19 patients is unclear. Some recovered patients complain of continued shortness of breath. Vasculopathy has been reported in COVID-19, stressing the importance of probing microstructure and function of lungs at the alveolar-capillary interface. While CT detects structural abnormalities, little is known about the impact of disease on lung function. 129Xe MRI is a technique uniquely capable of assessing ventilation, microstructure and gas exchange. Using 129Xe MRI, we found COVID-19 patients have higher ventilation defects percentage (5.9% vs 3.7%), unchanged microstructure, longer gas-blood exchange time (43.5 ms vs 32.5 ms), and reduced RBC/TP (0.279 vs 0.330) compared with healthy subjects. These findings suggest regional ventilation and alveolar airspace dimensions are relatively normal around the time of discharge, while gas-blood exchange function is diminished. This study establishes the feasibility of localized lung function measurement in COVID-19 patients. Such readouts could be useful as a supplement to structural imaging.

Lung adenocarcinoma (LUAD) harboring EGFR mutations prevails in Asian population. However, the inter-patient and intra-tumor heterogeneity has not been addressed at single-cell resolution. Here we performed single-cell RNA sequencing (scRNA-seq) of total 125,674 cells from seven stage-I/II LUAD samples harboring EGFR mutations and five tumor-adjacent lung tissues. We identified diverse cell types within the tumor microenvironment (TME) in which myeloid cells and T cells were the most abundant stromal cell types in tumors and adjacent lung tissues. Within tumors, accompanied by an increase in CD1C+ dendritic cells, the tumor-associated macrophages (TAMs) showed pro-tumoral functions without signature gene expression of defined M1 or M2 polarization. Tumor-infiltrating T cells mainly displayed exhausted and regulatory T-cell features. The adenocarcinoma cells can be categorized into different subtypes based on their gene expression signatures in distinct pathways such as hypoxia, glycolysis, cell metabolism, translation initiation, cell cycle, and antigen presentation. By performing pseudotime trajectory, we found that ELF3 was among the most upregulated genes in more advanced tumor cells. In response to secretion of inflammatory cytokines (e.g., IL1B) from immune infiltrates, ELF3 in tumor cells was upregulated to trigger the activation of PI3K/Akt/NF-κB pathway and elevated expression of proliferation and anti-apoptosis genes such as BCL2L1 and CCND1. Taken together, our study revealed substantial heterogeneity within early-stage LUAD harboring EGFR mutations, implicating complex interactions among tumor cells, stromal cells and immune infiltrates in the TME.

No abstract

As a new neural machine translation approach, Non-Autoregressive machine Translation (NAT) has attracted attention recently due to its high efficiency in inference. However, the high efficiency has come at the cost of not capturing the sequential dependency on the target side of translation, which causes NAT to suffer from two kinds of translation errors: 1) repeated translations (due to indistinguishable adjacent decoder hidden states), and 2) incomplete translations (due to incomplete transfer of source side information via the decoder hidden states). In this paper, we propose to address these two problems by improving the quality of decoder hidden representations via two auxiliary regularization terms in the training process of an NAT model. First, to make the hidden states more distinguishable, we regularize the similarity between consecutive hidden states based on the corresponding target tokens. Second, to force the hidden states to contain all the information in the source sentence, we leverage the dual nature of translation tasks (e.g., English to German and German to English) and minimize a backward reconstruction error to ensure that the hidden states of the NAT decoder are able to recover the source side sentence. Extensive experiments conducted on several benchmark datasets show that both regularization strategies are effective and can alleviate the issues of repeated translations and incomplete translations in NAT models. The accuracy of NAT models is therefore improved significantly over the state-of-the-art NAT models with even better efficiency for inference.

Non-autoregressive translation (NAT) models, which remove the dependence on previous target tokens from the inputs of the decoder, achieve significantly inference speedup but at the cost of inferior accuracy compared to autoregressive translation (AT) models. Previous work shows that the quality of the inputs of the decoder is important and largely impacts the model accuracy. In this paper, we propose two methods to enhance the decoder inputs so as to improve NAT models. The first one directly leverages a phrase table generated by conventional SMT approaches to translate source tokens to target tokens, which are then fed into the decoder as inputs. The second one transforms source-side word embeddings to target-side word embeddings through sentence-level alignment and word-level adversary learning, and then feeds the transformed word embeddings into the decoder as inputs. Experimental results show our method largely outperforms the NAT baseline (Gu et al. 2017) by 5.11 BLEU scores on WMT14 English-German task and 4.72 BLEU scores on WMT16 English-Romanian task.