A major obstacle in Word Sense Disambiguation (WSD) is that word senses are not uniformly distributed, causing existing models to generally perform poorly on senses that are either rare or unseen during training. We propose a bi-encoder model that independently embeds (1) the target word with its surrounding context and (2) the dictionary definition, or gloss, of each sense. The encoders are jointly optimized in the same representation space, so that sense disambiguation can be performed by finding the nearest sense embedding for each target word embedding. Our system outperforms previous state-of-the-art models on English all-words WSD; these gains predominantly come from improved performance on rare senses, leading to a 31.1% error reduction on less frequent senses over prior work. This demonstrates that rare senses can be more effectively disambiguated by modeling their definitions.

We present a set of experiments to demonstrate that deep recurrent neural networks (RNNs) learn internal representations that capture soft hierarchical notions of syntax from highly varied supervision. We consider four syntax tasks at different depths of the parse tree; for each word, we predict its part of speech as well as the first (parent), second (grandparent) and third level (great-grandparent) constituent labels that appear above it. These predictions are made from representations produced at different depths in networks that are pretrained with one of four objectives: dependency parsing, semantic role labeling, machine translation, or language modeling. In every case, we find a correspondence between network depth and syntactic depth, suggesting that a soft syntactic hierarchy emerges. This effect is robust across all conditions, indicating that the models encode significant amounts of syntax even in the absence of an explicit syntactic training supervision.

No abstract

The colored component of several important ancient pigments, including Egyptian blue and Han blue, are based on alkali earth copper tetrasilicate materials. In recent work, we have found that these layered materials can be chemically exfoliated into their constituent monolayers to provide alkali earth copper tetrasilicate nanosheets—defined by nanometer thickness and lateral dimensions that are on the order of several microns. The facile exfoliation of these materials into nanosheets is especially surprising in view of their long history on artifacts under a variety of environmental conditions, and we have examined the issue of whether archaeological samples are affected by this exfoliation mechanism. We have characterized the properties of these nanosheets by an array of analytical techniques, including powder x-ray diffraction, photoluminescence measurements, and Raman spectroscopy. In all cases, we observe differences between nanosheet and bulk samples that originate from the loss of coupling between layers when going from three-dimensional to two- dimensional structures. Both CaCuSi4O10 nanosheets (derived from Egyptian blue) and BaCuSi4O10 nanosheets (derived from Han blue) have strong near-infrared luminescence properties like their bulk counterparts, yet they are amenable to modern solution processing methods. We have demonstrated ink jet printing with CaCuSi4O10 nanosheet inks, as well as the fabrication of nanosheet-based papers. Potential applications for these materials include NIR-based biomedical imaging and security inks.

A major obstacle in Word Sense Disambiguation (WSD) is that word senses are not uniformly distributed, causing existing models to generally perform poorly on senses that are either rare or unseen during training. We propose a bi-encoder model that independently embeds (1) the target word with its surrounding context and (2) the dictionary definition, or gloss, of each sense. The encoders are jointly optimized in the same representation space, so that sense disambiguation can be performed by finding the nearest sense embedding for each target word embedding. Our system outperforms previous state-of-the-art models on English all-words WSD; these gains predominantly come from improved performance on rare senses, leading to a 31.1% error reduction on less frequent senses over prior work. This demonstrates that rare senses can be more effectively disambiguated by modeling their definitions.