An Effective Tokenization Algorithm for Information Retrieval Systems

Singh, Vikram; Saini, Balwinder

doi:10.5121/csit.2014.4910

Cited by 25 publications

(21 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…[1] Ontologies are metadata schemas, providing a controlled vocabulary of concepts, each with an explicitly defined and machine process able semantics [1] [7], by defining shared and common domain theories, ontology helps both people and machines to communicate precisely to support the exchange of semantics. Ontology language editors help to build semantic web [3][4] [5]. Hence, the contemptible and efficient construction of domain specific ontology is crucial for the success of many data processing systems.…”

Section: Introductionmentioning

confidence: 99%

Untitled

2014

IJDMS

View full text Add to dashboard Cite

Information is growing more rapidly on the World Wide Web (WWW) has made it necessary to make all this information not only available to people but also to the machines. Ontology and token are widely being used to add the semantics in data processing or information processing. A concept formally refers to the meaning of the specification which is encoded in a logic-based language, explicit means concepts, properties that specification is machine readable and also a conceptualization model how people think about things of a particular subject area. In modern scenario more ontologies has been developed on various different topics, results in an increased heterogeneity of entities among the ontologies. The concept integration becomes vital over last decade and a tool to minimize heterogeneity and empower the data processing. There are various techniques to integrate the concepts from different input sources, based on the semantic or syntactic match values. In this paper, an approach is proposed to integrate concept (Ontologies or Tokens) using edit distance or n-gram match values between pair of concept and concept frequency is used to dominate the integration process. The proposed techniques performance is compared with semantic similarity based integration techniques on quality parameters like Recall, Precision, F-Measure & integration efficiency over the different size of concepts. The analysis indicates that edit distance value based interaction outperformed n-gram integration and semantic similarity techniques.

show abstract

Section: Introductionmentioning

confidence: 99%

Untitled

2014

IJDMS

View full text Add to dashboard Cite

show abstract

“…The integration results into creation of merged node and best position for newly created will be based on highest values of frequency among participating concept. Precision is a value in the range [0, 1]; the higher the value, the fewer wrong merging computed [5] [11]. Precision is determines as the ration of number of correct found alignment/ matching with total number of alignment found.…”

Section: Proposed Procedures and Examplementioning

confidence: 99%

“…[1] Ontologies are metadata schemas, providing a controlled vocabulary of concepts, each with an explicitly defined and machine process able semantics [1][7], by defining shared and common domain theories, ontology helps both people and machines to communicate precisely to support the exchange of semantics. Ontology language editors help to build semantic web [3][4] [5]. Hence, the contemptible and efficient construction of domain specific ontology is crucial for the success of many data processing systems.…”

Section: Introductionmentioning

confidence: 99%

Untitled

2014

IJDMS

View full text Add to dashboard Cite

show abstract

“…These words, numbers, symbols and other characters distinguished by tokenization called tokens. [4].…”

Section: Introductionmentioning

confidence: 99%

“…Stop word removing is substantial in the preprocessing, it has some advantages like reducing the size of stored data set and it improves the overall efficiency and effectiveness of the analysis system. [4]. The proposed system uses a list of stop words obtained from Onix Text Retrieval Toolkit website [5].…”

Section: Introductionmentioning

confidence: 99%