Deep Learning with Logical Constraints

Abstract

As the final stage of the multi-stage recommender system (MRS), re-ranking directly affects users’ experience and satisfaction by rearranging the input ranking lists, and thereby plays a critical role in MRS. With the advances in deep learning, neural re-ranking has become a trending topic and been widely adopted in industrial applications. This review aims at integrating re-ranking algorithms into a broader picture, and paving ways for more comprehensive solutions for future research. For this purpose, we first present a taxonomy of current methods on neural re-ranking. Then we give a description of these methods along with the historic development according to their objectives. The network structure, personalization, and complexity are also discussed and compared. Next, we provide a benchmark for the major neural re-ranking models and quantitatively analyze their re-ranking performance. Finally, the review concludes with a discussion on future prospects of this field. A list of papers discussed in this review, the benchmark datasets, our re-ranking library LibRerank, and detailed parameter settings are publicly available at https://github.com/LibRerank-Community/LibRerank.

Refining neural network predictions using background knowledge

“…ILR falls into a larger body of work that attempts to integrate background knowledge expressed as logical formulas into neural networks. For an overview, see (Giunchiglia et al, 2022). Figure 1 shows two categories that most methods fall in.…”

“…ILR is a method in the second category. We note that these approaches can be combined (Giunchiglia et al, 2022a;Roychowdhury et al, 2021). First, we discuss approaches that construct loss functions from the logical formulas (Fig.…”

“…Therefore, we can not rely on efficient convex solvers. Furthermore, for strict t-norms, finding exact solutions to this problem is equivalent to solving PMaxSAT when t𝜑 = 1 (Diligenti et al, 2017;Giunchiglia et al, 2022a), hence this problem is NP-complete. In Sects.…”

“…Recent years have shown promising examples of using symbolic background knowledge in learning systems: From training classifiers with weak supervision signals (Manhaeve et al, 2018), generalizing learned classifiers to new tasks (Roychowdhury et al, 2021), compensating for a lack of good supervised data (Diligenti et al, 2017;Donadello et al, 2017), to enforcing the structure of outputs through a logical specification (Xu et al, 2018). The main idea underlying these integrations of learning and reasoning, often called neurosymbolic integration, is that background knowledge can complement the neural network when one lacks high-quality labeled data (Giunchiglia et al, 2022). Although pure deep learning approaches excel when learning over vast quantities of data with gigantic amounts of compute (Chowdhery et al, 2022;Ramesh et al, 2022), we cannot afford this luxury for most tasks.…”

“…1a). However, often the logic is discarded after training, even though this background knowledge could still be helpful at test time (Roychowdhury et al, 2021;Giunchiglia et al, 2022a). Instead, we ensure we constrain the neural network with the background knowledge, both during train time and test time, by correcting its output to satisfy the background knowledge (Fig.…”