Information Extraction Tasks based on BERT and SpaCy on Tourism Domain

Chantrapornchai, Chantana; Tunsakul, Aphisit

doi:10.37936/ecti-cit.2021151.228621

Cited by 16 publications

(13 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…spaCy is a free, open-source library for advanced Natural Language Processing (NLP) in Python [36,37]. spaCy supports over 64 language models trained on either newspaper articles, media, blogs, comments etc.…”

Section: Spacymentioning

confidence: 99%

“…For example, in the sentence "I have to Google why Google has a lot of employees", the model would correctly recognize the first Google as a verb and the latter as an organisation entity, despite being constructed in a similar manner. Because of the models' reliance on sentence syntax for entity extraction [37], it is able to extract entities regardless of spelling errors or presence of noisy elements within the entity. In our previous study [16] we showed that the syntax reliance of spaCy gives it a much higher recall as compared to the gazetteer based DBpedia tool.…”

Section: Spacymentioning

confidence: 99%

“…Since spaCy extracts entities based on entity predictions [37], it is able to distinguish geographical from non-geographical entities, for instance, disambiguating the word 'Turkey' as either, a country, animal, food or a person's name based on sentence syntax. However, since most geocoding services do not take into account a tweet's context, geographical location ambiguities remain a challenge [42][43][44].…”

Section: Location Extractionmentioning

confidence: 99%

See 2 more Smart Citations

Enhanced geocoding precision for location inference of tweet text using spaCy, Nominatim and Google Maps. A comparative analysis of the influence of data selection

2023

View full text Add to dashboard Cite

Twitter location inference methods are developed with the purpose of increasing the percentage of geotagged tweets by inferring locations on a non-geotagged dataset. For validation of proposed approaches, these location inference methods are developed on a fully geotagged dataset on which the attached Global Navigation Satellite System coordinates are used as ground truth data. Whilst a substantial number of location inference methods have been developed to date, questions arise pertaining the generalizability of the developed location inference models on a non-geotagged dataset. This paper proposes a high precision location inference method for inferring tweets’ point of origin based on location mentions within the tweet text. We investigate the influence of data selection by comparing the model performance on two datasets. For the first dataset, we use a proportionate sample of tweet sources of a geotagged dataset. For the second dataset, we use a modelled distribution of tweet sources following a non-geotagged dataset. Our results showed that the distribution of tweet sources influences the performance of location inference models. Using the first dataset we outweighed state-of-the-art location extraction models by inferring 61.9%, 86.1% and 92.1% of the extracted locations within 1 km, 10 km and 50 km radius values, respectively. However, using the second dataset our precision values dropped to 45.3%, 73.1% and 81.0% for the same radius values.

show abstract

Section: Spacymentioning

confidence: 99%

Section: Spacymentioning

confidence: 99%

Section: Location Extractionmentioning

confidence: 99%

See 1 more Smart Citation

Enhanced geocoding precision for location inference of tweet text using spaCy, Nominatim and Google Maps. A comparative analysis of the influence of data selection

2023

View full text Add to dashboard Cite

show abstract

“…Specifically, empirical results shown by transformers dramatically outperform the classical pipeline of machine learning models with a bag-of-words representation of the most common and relevant words of the texts according to algorithms such as TF-IDF [16]. Driven by the amazing results of transformer models, several communities like finance [37], energy [8] or tourism [9] are starting to use them for a wide variety of tasks.…”

Section: Fundamentals Of Large Language Models For Supervised Classif...mentioning

confidence: 99%

Fine-tuning ClimateBert transformer with ClimaText for the disclosure analysis of climate-related financial risks

Garrido-Merchán¹,

González-Barthe²,

Vaca³

2023

Preprint

View full text Add to dashboard Cite

In recent years there has been a growing demand from financial agents, especially from particular and institutional investors, for companies to report on climate-related financial risks. A vast amount of information, in text format, can be expected to be disclosed in the short term by firms in order to identify these types of risks in their financial and non financial reports, particularly in response to the growing regulation that is being passed on the matter. To this end, this paper applies stateof-the-art NLP techniques to achieve the detection of climate change in text corpora. We use transfer learning to fine-tune two transformer models, BERT and ClimateBert -a recently published DistillRoBERTa-based model that has been specifically tailored for climate text classification-. These two algorithms are based on the transformer architecture which enables learning the contextual relationships between words in a text. We carry out the fine-tuning process of both models on the novel "Clima-Text" database, consisting of data collected from Wikipedia, 10K Files Reports and web-based claims. Our text classification model obtained from the ClimateBert fine-tuning process on ClimaText, outperforms the models created with BERT and the current state-of-the-art transformer in this particular problem. Our study is the first one to implement on the ClimaText database the recently published ClimateBert algorithm. Based on our results, it can be said that ClimateBert fine-tuned on Cli-maText is an outstanding tool within the NLP pre-trained transformer models that may and should be used by investors, institutional agents and companies themselves to monitor the disclosure of climate risk in financial reports, as well as in other textual sources, such as policies, new legislation or directives passed on this matter. In addition, our transfer learning methodology is cheap in computational terms, thus allowing any organization to perform it.

show abstract

“…For example, in tourism domain, NER is used to label the point of interest, reviews, hotels, location, etc. (Chantrapornchai and Tunsakul, 2021); in medical domain, NER identifies clinical entities in electronic medical records and assigns them to previously defined categories, such as disease, image review, laboratory examination, operation, drug and anatomy (Kong et al , 2021); in commerce domain, NER is introduced to identify entity name of the cross border e-commerce commodity (Luo et al , 2020); in architecture domain, NER also successfully recognizes bridge names, structural members, member elements, locations of members or elements, structural defects and negative descriptions in bridge inspection reports (Li et al , 2021).…”

Section: Related Workmentioning

confidence: 99%

Identifying business information through deep learning: analyzing the tender documents of an Internet-based logistics bidding platform

2023

DTA

View full text Add to dashboard Cite

PurposeThe tender documents, an essential data source for internet-based logistics tendering platforms, incorporate massive fine-grained data, ranging from information on tenderee, shipping location and shipping items. Automated information extraction in this area is, however, under-researched, making the extraction process a time- and effort-consuming one. For Chinese logistics tender entities, in particular, existing named entity recognition (NER) solutions are mostly unsuitable as they involve domain-specific terminologies and possess different semantic features.Design/methodology/approachTo tackle this problem, a novel lattice long short-term memory (LSTM) model, combining a variant contextual feature representation and a conditional random field (CRF) layer, is proposed in this paper for identifying valuable entities from logistic tender documents. Instead of traditional word embedding, the proposed model uses the pretrained Bidirectional Encoder Representations from Transformers (BERT) model as input to augment the contextual feature representation. Subsequently, with the Lattice-LSTM model, the information of characters and words is effectively utilized to avoid error segmentation.FindingsThe proposed model is then verified by the Chinese logistic tender named entity corpus. Moreover, the results suggest that the proposed model excels in the logistics tender corpus over other mainstream NER models. The proposed model underpins the automatic extraction of logistics tender information, enabling logistic companies to perceive the ever-changing market trends and make far-sighted logistic decisions.Originality/value(1) A practical model for logistic tender NER is proposed in the manuscript. By employing and fine-tuning BERT into the downstream task with a small amount of data, the experiment results show that the model has a better performance than other existing models. This is the first study, to the best of the authors' knowledge, to extract named entities from Chinese logistic tender documents. (2) A real logistic tender corpus for practical use is constructed and a program of the model for online-processing real logistic tender documents is developed in this work. The authors believe that the model will facilitate logistic companies in converting unstructured documents to structured data and further perceive the ever-changing market trends to make far-sighted logistic decisions.

show abstract

Information Extraction Tasks based on BERT and SpaCy on Tourism Domain

Cited by 16 publications

References 10 publications

Enhanced geocoding precision for location inference of tweet text using spaCy, Nominatim and Google Maps. A comparative analysis of the influence of data selection

Enhanced geocoding precision for location inference of tweet text using spaCy, Nominatim and Google Maps. A comparative analysis of the influence of data selection

Fine-tuning ClimateBert transformer with ClimaText for the disclosure analysis of climate-related financial risks

Identifying business information through deep learning: analyzing the tender documents of an Internet-based logistics bidding platform

Contact Info

Product

Resources

About