Transformer models have advanced the state of the art in many Natural Language Processing (NLP) tasks.In this paper, we present a new Transformer architecture, Extended Transformer Construction (ETC), that addresses two key challenges of standard Transformer architectures, namely scaling input length and encoding structured inputs. To scale attention to longer inputs, we introduce a novel global-local attention mechanism between global tokens and regular input tokens. We also show that combining global-local attention with relative position encodings and a Contrastive Predictive Coding (CPC) pretraining objective allows ETC to encode structured inputs. We achieve state-of-the-art results on four natural language datasets requiring long and/or structured inputs.

No abstract

No abstract

Tofu is a traditional product made mainly from soybeans, which has become globally popular because of its inclusion in vegetarian, vegan, and hypocaloric diets. However, with both commercial production of tofu and scientific research, it remains a challenge to produce tofu with high quality, high nutrition, and excellent flavor. This is because tofu production involves multiple complicated steps, such as soybean selection, utilization of appropriate coagulants, and tofu packaging. To make high-quality tofu product, it is important to systematically understand critical factors that influence tofu quality. This article reviews the current research status of tofu production. The diversity of soybean seeds (the raw material), protein composition, structural properties, and nutritional values are reviewed. Then, selection of tofu coagulants is reviewed to provide insights on its role in tofu quality, where the focus is on the usage of mix coagulants and recent developments with new coagulants. Moreover, a comprehensive summary is provided on recent development in making high-fiber tofu using Okara (the major by-product during tofu production), which has a number of potential applications in the food industry. To help encourage automatic, environmental friendly, and high-efficient tofu production, new developments and applications in production technology, such as ultrasound and high-pressure process, are reviewed. Tofu packaging, including packaging materials and techniques, is evaluated as it has been found to have a positive impact on extending the shelf life and improving the quality of tofu products. Finally, the future research directions and potential areas for new developments are discussed.

No abstract

2016-11-03T14:11:40

Attribute value extraction refers to the task of identifying values of an attribute of interest from product information. It is an important research topic which has been widely studied in e-Commerce and relation learning. There are two main limitations in existing attribute value extraction methods: scalability and generalizability. Most existing methods treat each attribute independently and build separate models for each of them, which are not suitable for large scale attribute systems in real-world applications. Moreover, very limited research has focused on generalizing extraction to new attributes. In this work, we propose a novel approach for Attribute Value Extraction via Question Answering (AVEQA) using a multi-task framework. In particular, we build a question answering model which treats each attribute as a question and identifies the answer span corresponding to the attribute value in the product context. A unique BERT contextual encoder is adopted and shared across all attributes to encode both the context and the question, which makes the model scalable. A distilled masked language model with knowledge distillation loss is introduced to improve the model generalization ability. In addition, we employ a no-answer classifier to explicitly handle the cases where there are no values for a given attribute in the product context. The question answering, distilled masked language model and the no answer classification are then combined into a unified multi-task framework. We conduct extensive experiments on a public dataset. The results demonstrate that the proposed approach outperforms several state-of-the-art methods with large margin.

No abstract