Searching and Browsing Linked Data with SWSE: The Semantic Web Search Engine

Abstract

In this paper, we discuss the architecture and implementation of the Semantic Web Search Engine (SWSE). Following traditional search engine architecture, SWSE consists of crawling, data enhancing, indexing and a user interface for search, browsing and retrieval of information; unlike traditional search engines, SWSE operates over RDF Web dataloosely also known as Linked Data -which implies unique challenges for the system design, architecture, algorithms, implementation and user interface. In particular, many challenges exist in adopting Semantic Web technologies for Web data: the unique challenges of the Web -in terms of scale, unreliability, inconsistency and noise -are largely overlooked by the current Semantic Web standards. Herein, we describe the current SWSE system, initially detailing the architecture and later elaborating upon the function, design, implementation and performance of each individual component. In so doing, we also give an insight into how current Semantic Web standards can be tailored, in a besteffort manner, for use on Web data. Throughout, we offer evaluation and complementary argumentation to support our design choices, and also offer discussion on future directions and open research questions. Later, we also provide candid discussion relating to the difficulties currently faced in bringing such a search engine into the mainstream, and lessons learnt from roughly six years working on the Semantic Web Search Engine project.

Ontology lexicalization: Relationship between content and meaning in the context of Information Retrieval

“…Exploring the Linked Open Data (LOD) potential, Hogan et al (2011) proposed a Semantic Web Search Engine (SWSE) for searching and browsing RDF Web data. Given the flexibility of the semantic web, retrieved objects can represent people, companies, cities, proteins, or anything that has been published without predefined categorization such as that in traditional search engines.…”

Robust and scalable Linked Data reasoning incorporating provenance and trust annotations

“…One particular use-case we have in mind for including the blacklisting annotation relates to the publication of void values for inverse-functional properties: the Friend Of A Friend (FOAF) 14 vocabulary offers classes and properties for describing information about people, organisations, documents, and so forth: it is currently one of the most widely instantiated vocabularies in Linked Data [37,Appendix A]. FOAF includes a number of inverse-functional-properties which allow for identifying (esp.)…”

“…In the following sections, we look at the design, implementation and evaluation of (i) ranking (Section 6); (ii) annotated reasoning (Section 7); (iii) inconsistency detection and repair using constraint rules (Section 8). Before we continue, in this section we describe the distributed framework (see [37]) which forms the substrate of our experiments, and then describe our experimental Linked Data corpus.…”

“…26 Also, all machines have a local copy of a sorted list of redirects encountered during the crawl: i.e., ordered pairs of the form f, t where redir(f ) = t. The input data are unsorted, not uniqued, and predistributed over eight slave machines according to a hash function of context; this is the direct result of our distributed crawl, details of which are available in [37].…”

“…4 We have previously demonstrated distributed reasoning over ∼1b Linked Data triples [38], and have also presented some preliminary formalisations of what we call "authoritative reasoning", which considers the source of terminological data during reasoning [36]. The SAOR system is actively used for materialising inferences in the Semantic Web Search Engine (SWSE) [37] which offers search and browsing over Linked Data.…”