Geo-knowledge-guided GPT models improve the extraction of location descriptions from disaster-related social media messages

Abstract

Social media messages posted by people during natural disasters often contain important location descriptions, such as the locations of victims. Recent research has shown that many of these location descriptions go beyond simple place names, such as city names and street names, and are difficult to extract using typical named entity recognition (NER) tools. While advanced machine learning models could be trained, they require large labeled training datasets that can be time-consuming and labor-intensive to create. In this work, we propose a method that fuses geoknowledge of location descriptions and a Generative Pre-trained Transformer (GPT) model, such as ChatGPT and GPT-4. The result is a geo-knowledge-guided GPT model that can accurately extract location descriptions from disaster-related social media messages. Also, only 22 training examples encoding geo-knowledge are used in our method. We conduct experiments to compare this method with nine alternative approaches on a dataset of tweets from Hurricane Harvey. Our method demonstrates an over 40% improvement over typically used NER approaches. The experiment results also show that geo-knowledge is indispensable for guiding the behavior of GPT models. The extracted location descriptions can help disaster responders reach victims more quickly and may even save lives.

Generative Pre-Trained Transformer (GPT) in Research: A Systematic Review on Data Augmentation

“…The study demonstrates significant improvements in macro-averaged F1 scores on hate speech corpora using the augmented data. The study in [51] explores the use of GPT models for generating synthetic data to enhance machine learning applications. It emphasizes the creation of diverse and representative synthetic data to improve machine learning model robustness.…”

“…Synthetic Data Creation: Focuses on using GPT models to generate artificial data that mimics real-world data, useful in scenarios where data privacy is crucial or actual data is limited. Literature in [31,35,51,52,56,57] could be attributed to this sub-category. • Text Data Expansion and Enhancement: This involves leveraging GPT to create new textual content and enhance existing datasets, thereby improving machine learning models' performance and addressing data scarcity [12,26,37,39,41,48,53,54,65].…”

A Sustainable Way Forward: Systematic Review of Transformer Technology in Social-Media-Based Disaster Analytics

“…Studies in this category try to extract location descriptions and identify geographical references from disaster-related social media messages [49][50][51]. Table 7 summarizes the categorization of location identification and description extraction.…”

“…Integrating geo-knowledge with GPT models like ChatGPT and GPT-4 significantly improves location description extraction accuracy from social media, outperforming traditional NER approaches by over 40% [50]. Yet, its effectiveness depends on the availability and quality of geo-knowledge, posing challenges in generalization across regions and disaster types.…”

“…Yet, its effectiveness depends on the availability and quality of geo-knowledge, posing challenges in generalization across regions and disaster types. GPT-based [50] Significant improvement in accuracy of location extraction from social media messages by leveraging GPT models.…”