Neural Ranking Models with Weak Supervision

Dehghani, Mostafa; Zamani, Hamed; Severyn, Aliaksei; Kamps, Jaap; Croft, W. Bruce

doi:10.1145/3077136.3080832

Cited by 295 publications

(333 citation statements)

References 35 publications

Supporting

Mentioning

327

Contrasting

Order By: Relevance

“…We consider the task also as MRS because they share almost the same setup except that the downstream task is verification or natural language inference (NLI) rather than QA. Information Retrieval Success in deep neural networks inspires their application to information retrieval (IR) tasks (Huang et al, 2013;Guo et al, 2016;Mitra et al, 2017;Dehghani et al, 2017). In typical IR settings, systems are required to retrieve and rank (Nguyen et al, 2016) elements from a collection of documents based on their relevance to the query.…”

Section: Related Workmentioning

confidence: 99%

Revealing the Importance of Semantic Retrieval for Machine Reading at Scale

Nie¹,

Wang²,

Bansal³

2019

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conferen

116

103

View full text Add to dashboard Cite

Machine Reading at Scale (MRS) is a challenging task in which a system is given an input query and is asked to produce a precise output by "reading" information from a large knowledge base. The task has gained popularity with its natural combination of information retrieval (IR) and machine comprehension (MC). Advancements in representation learning have led to separated progress in both IR and MC; however, very few studies have examined the relationship and combined design of retrieval and comprehension at different levels of granularity, for development of MRS systems. In this work, we give general guidelines on system design for MRS by proposing a simple yet effective pipeline system with special consideration on hierarchical semantic retrieval at both paragraph and sentence level, and their potential effects on the downstream task. The system is evaluated on both fact verification and open-domain multihop QA, achieving state-of-the-art results on the leaderboard test sets of both FEVER and HOTPOTQA. To further demonstrate the importance of semantic retrieval, we present ablation and analysis studies to quantify the contribution of neural retrieval modules at both paragraph-level and sentence-level, and illustrate that intermediate semantic retrieval modules are vital for not only effectively filtering upstream information and thus saving downstream computation, but also for shaping upstream data distribution and providing better data for downstream modeling. 1

show abstract

Section: Related Workmentioning

confidence: 99%

Revealing the Importance of Semantic Retrieval for Machine Reading at Scale

Nie¹,

Wang²,

Bansal³

2019

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conferen

116

103

View full text Add to dashboard Cite

show abstract

“…Parameters and training. We train the neural models using pairwise crossentropy loss [3]. Hyper-parameters are tuned using nCDG@5 on the dev set.…”

Section: Methodsmentioning

confidence: 99%

Ranking Significant Discrepancies in Clinical Reports

MacAvaney

Cohan

Goharian

et al. 2020

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Medical errors are a major public health concern and a leading cause of death worldwide. Many healthcare centers and hospitals use reporting systems where medical practitioners write a preliminary medical report and the report is later reviewed, revised, and finalized by a more experienced physician. The revisions range from stylistic to corrections of critical errors or misinterpretations of the case. Due to the large quantity of reports written daily, it is often difficult to manually and thoroughly review all the finalized reports to find such errors and learn from them. To address this challenge, we propose a novel ranking approach, consisting of textual and ontological overlaps between the preliminary and final versions of reports. The approach learns to rank the reports based on the degree of discrepancy between the versions. This allows medical practitioners to easily identify and learn from the reports in which their interpretation most substantially differed from that of the attending physician (who finalized the report). This is a crucial step towards uncovering potential errors and helping medical practitioners to learn from such errors, thus improving patient-care in the long run. We evaluate our model on a dataset of radiology reports and show that our approach outperforms both previously-proposed approaches and more recent language models by 4.5% to 15.4%.

show abstract

“…Furthermore, it is possible to use heuristic methods to generate weak supervision signals and to go beyond them by employing proper learning objectives and network designs [14].…”

Section: Objectivesmentioning

confidence: 99%

Neural Networks for Information Retrieval

Kenter¹,

Borisov

Gysel

et al. 2018

Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining

Self Cite

View full text Add to dashboard Cite

Machine learning plays a role in many aspects of modern IR systems, and deep learning is applied in all of them. The fast pace of modernday research has given rise to many di erent approaches for many di erent IR problems. The amount of information available can be overwhelming both for junior students and for experienced researchers looking for new research topics and directions. Additionally, it is interesting to see what key insights into IR problems the new technologies are able to give us. The aim of this full-day tutorial is to give a clear overview of current tried-and-trusted neural methods in IR and how they bene t IR research. It covers key architectures, as well as the most promising future directions. MOTIVATIONPrompted by the advances of deep learning in computer vision research, neural networks have resurfaced as a popular machine learning paradigm in many other directions of research as well, including information retrieval. Recent years have seen neural networks being applied to all key parts of the typical modern IR pipeline, such core ranking algorithms [26,42,51], click models [9,10], knowledge graphs [8,35], text similarity [28,47], entity retrieval [52,53], language modeling [5], question answering [22,56], and dialogue systems [34,54].A key advantage that sets neural networks apart from many learning strategies employed earlier, is their ability to work from raw input data. E.g., when given enough training data, well-designed networks can become feature extractors themselves, e.g., incorporating basic input characteristics such as term frequency (tf) and term saliency (idf)-that used to be pre-calculated o ine-in their initial layers. Where designing features used to be a crucial aspect and contribution of newly proposed IR approaches, the focus * Corresponding author.SIGIR '17, Shinjuku, Tokyo, Japan has shifted to designing network architectures instead. As a consequence, many di erent architectures and paradigms have been proposed, such as auto-encoders, recursive networks, recurrent networks, convolutional networks, various embedding methods, deep reinforcement and deep q-learning, and, more recently, generative adversarial networks, of which most have been applied in IR settings. The aim of the neural networks for IR (NN4IR) tutorial is to provide a clear overview of the main network architectures currently applied in IR and to show explicitly how they relate to previous work. The tutorial covers methods applied in industry and academia, with in-depth insights into the underlying theory, core IR tasks, applicability, key assets and handicaps, scalability concerns and practical tips and tricks.We expect the tutorial to be useful both for academic and industrial researchers and practitioners who either want to develop new neural models, use them in their own research in other areas or apply the models described here to improve actual IR systems. OBJECTIVESThe material in the tutorial covers a broad range of IR applications. It is structured as follows:Preliminaries (60 minutes). The rece...

show abstract

Neural Ranking Models with Weak Supervision

Cited by 295 publications

References 35 publications

Revealing the Importance of Semantic Retrieval for Machine Reading at Scale

Revealing the Importance of Semantic Retrieval for Machine Reading at Scale

Ranking Significant Discrepancies in Clinical Reports

Neural Networks for Information Retrieval

Contact Info

Product

Resources

About