Semantic Hilbert Space for Text Representation Learning

Abstract

Capturing the meaning of sentences has long been a challenging task. Current models tend to apply linear combinations of word features to conduct semantic composition for bigger-granularity units e.g. phrases, sentences, and documents. However, the semantic linearity does not always hold in human language. For instance, the meaning of the phrase "ivory tower" cannot be deduced by linearly combining the meanings of "ivory" and "tower". To address this issue, we propose a new framework that models different levels of semantic units (e.g. sememe, word, sentence, and semantic abstraction) on a single Semantic Hilbert Space, which naturally admits a non-linear semantic composition by means of a complex-valued vector word representation. An end-to-end neural network 1 is proposed to implement the framework in the text classification task, and evaluation results on six benchmarking text classification datasets demonstrate the effectiveness, robustness and self-explanation power of the proposed model. Furthermore, intuitive case studies are conducted to help end users to understand how the framework works. CCS CONCEPTS• Information systems → Document structure; Content analysis and feature selection; KEYWORDS text understanding, neural network, quantum theory ACM Reference Format:

Quantum-inspired multimodal fusion for video sentiment analysis

“…Zhang et al [51] build neural networks to learn high-level interactions of question and answer based on their entangled state representations for question answering (QA). [22,21,20] leverage quantum superposition and mixture to model correlations between linguistic features and construct complex-valued language representations by neural networks. The representation leads to improved performance and enhanced interpretability.…”

“…The inspiration stems from the manifestation of non-classical phenomena in human cognition and decision, which violates classical probability theory but adopts a compact explanation via quantum theory (QT) [18]. QT has stimulated the successful construction of quantuminspired models for human cognition-related tasks, such as information retrieval (IR) [19,20] and language understanding [21,22]. As a typical human cognitive task, however, multimodal sentiment analysis has received little at-tention from a quantum-inspired viewpoint [23,24], due to the challenge in modeling complicated interactions across different modalities in a quantum manner.…”

QSAN: A Quantum-probability based Signed Attention Network for Explainable False Information Detection

“…Complex-valued text representation has shown promising performance in NLP tasks such as text classification [42], information retrieval [31] and question answering [23]. Complex embeddings are also integrated into NN-architectures such as Transformer [47], convolutional feed-forward networks and convolutional LSTMs [40], but they don't follow the quantum operations such as mixture and measurement.…”

“…[7] proposes a mathematical framework for modeling semantic feature spaces using quantum mechanics, and verifies that implicit semantic analysis and HAL (Hyperspace Analog to Language) models are essentially standardized Hilbert spaces. [42] applies quantum theory to text representation learning tasks. [23] implements a quantum probability-based model for the QA problem.…”

“…Inspired by [42], we develop the quantum probability-driven encoder over the Semantic Hilbert Space, aiming to capture the nonlinear representation of each sentence and comment. Thus, the semantics of sentences and comments are expressed more comprehensively, and better prerequisites are provided for signed attention modules to extract the semantic relationship between them.…”