End to End Neural Retrieval for Patent Prior Art Search

Abstract

This research will examine neural retrieval methods for patent prior art search. One research direction is the federated search approach, where we proposed two new methods that solve the results merging problem in federated patent search using machine learning models. The methods are based on a centralized index containing samples of documents from all potential resources, and they implement machine learning models to predict comparable scores for the documents retrieved by different resources. The other research direction is the adaptation of end-to-end neural retrieval approaches to the patent characteristics such that the retrieval effectiveness will be increased. Off-the-self neural methods like BERT have lower effectiveness for patent prior art search. So, we adapt the BERT model to patent characteristics in order to increase retrieval performance. We propose a new gate-based document retrieval method and examine it in patent prior art search. The method combines a first-stage retrieval method using BM25 and a re-ranking approach where the BERT model is used as a gating function that operates on the BM25 score and modifies it according to the BERT relevance score. These experiments are based on two-stage retrieval approaches as neural models like BERT requires lots of computing power to be used. Eventually, the final part of the research will examine first-stage neural retrieval methods such as dense retrieval methods adapted to patent characteristics for prior art search.

A Review on the Determination of Semantic Similarity of Patent Documents

“…[50] Benzer şekilde BM25 ve BERT modelinin iki aşamalı olarak kullanıldığı bir başka çalışmada da patent tekniğin bilinen durumu araştırmasında güzel sonuçlar elde edilmiştir. [51] Patent metinlerini sınıflandırma üzerinde çalışılan bir makalede, patent metinlerinden özellik çıkarımının zorluklarına değinilmiş ve bir sözlük ağı oluşturularak bu zorluklar aşılmaya çalışılmıştır. Word2Vec, CNN, LSTM ve BERT modelleri ile sözlük ağı bir arada kullanılmış ve en başarılı sonuçlar BERT modeli ile elde edilmiştir.…”