Development of information technology of term extraction from documents in natural language

Kungurtsev, Oleksii; Zinovatna, Svitlana; Potochniak, Iana; Kutasevych, Maxim

doi:10.15587/1729-4061.2018.147978

Cited by 2 publications

(3 citation statements)

References 7 publications

(13 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, the authors limited themselves to only one-word and two-word terms. At the same time, research has shown that terms can contain up to five or more words [11]. In [12], a general approach to the analysis of texts in natural language was proposed, but the problems of practical implementation of procedures for extracting terms and determining their labor intensity remained unresolved.…”

Section: Literature Review and Problem Statementmentioning

confidence: 99%

“…In [12], a general approach to the analysis of texts in natural language was proposed, but the problems of practical implementation of procedures for extracting terms and determining their labor intensity remained unresolved. The most noteworthy is the method that combines statistical processing of texts with partial syntactic analysis [11]. The method is based on forming a possible term from a noun and surrounding words.…”

Section: Literature Review and Problem Statementmentioning

confidence: 99%

“…In addition, the corpus of the studied documents was limited to documents from the area of the executive and legislative powers, which made it possible to provide a certain dynamism of the headings, but could not be extended to other domains. The dependence of errors in the selection of verbose terms on the number of words in documents in Slavic languages is determined experimentally in [11] (Table 1). Here, the loss of a term means a one-time appearance of some noun in a document, since in this case there is no way to build a verbose term on its basis.…”

Section: Literature Review and Problem Statementmentioning

confidence: 99%

See 2 more Smart Citations

Development of methods for pre-clustering and virtual merging of short documents for building domain dictionaries

Kungurtsev¹,

Zinovatna²,

Potochniak³

et al. 2020

EEJET

Self Cite

View full text Add to dashboard Cite

The aim of research is to improve the quality of domain dictionaries by expanding the corpus of the documents under study by using short documents. A document model is proposed that allows to define a short document and the need to combine it with other documents to highlight verbose terms. An algorithm for highlighting the substantive part of the document has been developed, since in a short document the heading and closing parts usually contain terms that are not related to the studied domain. A method for preliminary clustering of short documents to highlight verbose terms has been developed. The method is based on highlighting and counting occurrences of nouns (one-word terms) for all analyzed documents. The concept of document proximity is introduced, which is determined by the combination of two criteria: the relative number of matching terms and the relative frequency of occurrence of matching terms. The principle of grouping documents at the customer's site often does not correspond to the principles of grouping necessary for building a dictionary of the domain. In a short document, it is usually impossible to isolate a verbose term because the repetition of terms is very low. A method has been developed for virtual combining of short documents based on the principle of achieving the necessary repeatability of one-word terms. The merged document has the highest possible frequency of terms for the cluster it belongs to. At the same time, the original text of documents is preserved and the ability to associate the selected verbose term with those documents in which it is included. The experiment made it possible to find the best ratio for the elements of the document proximity coefficient and confirm the effectiveness of the proposed preliminary clustering method

show abstract

Section: Literature Review and Problem Statementmentioning

confidence: 99%

Section: Literature Review and Problem Statementmentioning

confidence: 99%

Section: Literature Review and Problem Statementmentioning

confidence: 99%

See 1 more Smart Citation