UTexas: Natural Language Semantics using Distributional Semantics and Probabilistic Logic

Abstract

We represent natural language semantics by combining logical and distributional information in probabilistic logic. We use Markov Logic Networks (MLN) for the RTE task, and Probabilistic Soft Logic (PSL) for the STS task. The system is evaluated on the SICK dataset. Our best system achieves 73% accuracy on the RTE task, and a Pearson's correlation of 0.71 on the STS task.

SemEval-2014 Task 1: Evaluation of Compositional Distributional Semantic Models on Full Sentences through Semantic Relatedness and Textual Entailment

“…In the table, systems in bold are those for which the authors submitted a paper (Ferrone and Zanzotto, 2014;Bjerva et al, 2014;Beltagy et al, 2014;Lai and Hockenmaier, 2014;Alves et al, 2014;León et al, 2014;Bestgen, 2014;Zhao et al, 2014;Vo et al, 2014;Biçici and Way, 2014;Lien and Kouylekov, 2014;Jimenez et al, 2014;Proisl and Evert, 2014;Gupta et al, 2014). For the others, we used the brief description sent with the system's results, double-checking the information with the authors.…”

Probabilistic Soft Logic for Semantic Textual Similarity

“…Inference in MLNs is theoretically intractable in the general case, and existing approximate inference algorithms are computationally expensive and sometimes inaccurate. Consequently, the MLN approach of Beltagy et al (2013) was unable to scale to long sentences and was only tested on the relatively short sentences in the Microsoft video description corpus used for STS (Agirre et al, 2012). On the other hand, inference in PSL reduces to a linear programming problem, which is theoretically and practically much more efficient.…”

“…In this paper, we use the same combined logicbased and distributional framework as Beltagy et al, (2013) but replace Markov Logic Networks with Probabilistic Soft Logic (PSL) (Kimmig et al, 2012;Bach et al, 2013). PSL is a probabilistic logic framework designed to have efficient inference.…”

“…For evaluation, we test on three STS datasets, and compare our PSL system with the MLN approach of Beltagy et al, (2013) and with distributional-only baselines. Experimental results demonstrate that, overall, PSL models human similarity judgements more accurately than these alternative approaches, and is significantly faster than MLNs.…”

“…1 Interestingly, this characterization echoes the principle of compositionality, which states that the meaning of a phrase is uniquely determined by the meaning of its parts and the rules that connect those parts. Beltagy et al (2013) proposed a hybrid approach to sentence similarity: They use a very deep representation of sentence meaning, expressed in first-order logic, to capture sentence structure, but combine it with distributional similarity ratings at the word and phrase level. Sentence similarity is then modelled as mutual entailment in a probabilistic logic.…”