Question answering method for infrastructure damage information retrieval from textual data using bidirectional encoder representations from transformers

Kim, Young S.; Bang, Su Sik; Sohn, Jiu; Kim, Hyoungkwan

doi:10.1016/j.autcon.2021.104061

Cited by 27 publications

(6 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Researchers have extensively investigated word embedding techniques, including Word2Vec (Mikolov et al, 2013) [17] and FastText (Joulin et al, 2017), which enable machines to understand the contextual relationships between words and phrases. Semantic search, powered by these embeddings, has been applied in various domains, including finance, for information retrieval and questionanswering (Chaudhuri et al, 2020) [18].…”

Section: A Review Of Previous Researchmentioning

confidence: 99%

RETRACTED: Interpreting, Querying, and Securing Complex Financial Documents: A Blockchain-Enabled, OCR & Language Model Approach with Enhanced Cybersecurity Features

2023

Preprint

View full text Add to dashboard Cite

Background: Financial documents are inherently complex and sensitive, necessitating sophisticated tools for their in- terpretation, querying, and securing. Accessing and interpret- ing crucial information within these documents is vital for decision-making but often proves challenging due to their intricate structure and the sensitive nature of the data they contain. Ensuring robust security and confidentiality for this data is paramount, prompting the need for advanced, integrated solutions. Methodology Overview: To address these challenges, the ”Bank-AI” application is introduced as a comprehensive tool integrating Optical Character Recognition (OCR) technology and advanced language models within a secure, blockchain- enabled framework. The OCR component is precisely engi- neered to extract text and data from diverse financial docu- ments accurately. Meanwhile, state-of-the-art language models are employed to facilitate intelligent and efficient querying and interpretation of the extracted data, providing users with insightful and actionable information. The system’s security is significantly enhanced through blockchain technology, offering tamper-proof, secure handling, and management of sensitive financial data, with advanced cybersecurity features ensuring data integrity and confidentiality. Results: Early evaluations indicate that the “Bank-AI” application markedly improves the efficiency of extracting, interpreting, and securing information from complex financial documents. It not only expedites operational workflows but also fortifies the security protocols safeguarding sensitive financial data. The application shows promising efficiency, accuracy of app is 88% and 90% respectively over time for different query types and potential in revolutionizing the management and analysis of financial documents, making it an invaluable asset for businesses and financial analysts alike. Future Work: While this study establishes a solid ground- work for the ”Bank-AI” application, there is ample scope for further research and development. Future work may explore optimization strategies for enhancing the application’s perfor- mance across various financial document types and domains. Investigating its scalability and adaptability to different finan- cial environments and extending its capabilities to handle a broader array of document structures and formats will also be crucial in future development phases.

show abstract

Section: A Review Of Previous Researchmentioning

confidence: 99%

RETRACTED: Interpreting, Querying, and Securing Complex Financial Documents: A Blockchain-Enabled, OCR & Language Model Approach with Enhanced Cybersecurity Features

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…The evaluation process is systematically executed for each of the models under consideration, specifically RoBERTa and IndoBERT, engendering a meticulous examination of their performance nuances. The evaluation metrics employed encompass both Exact Match and F1-Score values, indicative of the models' precision and overall accuracy in generating responses to posed questions [27]. A visual representation of the intricate evaluation process flow is encapsulated in Figure 3.…”

Section: F Evaluationmentioning

confidence: 99%

Comparative Analysis of Transformer-Based Method In A Question Answering System for Campus Orientation Guides

Dartiko,

Yusa,

Erlansari

et al. 2024

intensif

View full text Add to dashboard Cite

The campus introduction process is a stage where new students acquire information about the campus through a series of activities and interactions with existing students. However, the delivery of campus introduction information is still limited to conventional methods, such as using guidebooks. This limitation can result in students having a limited understanding of the information needed during their academic period. The one of solution for this case is to implement a deep learning system with knowledge-based foundations. This research aims to develop a Question Answering System (QAS) as a campus introduction guide by comparing two transformer methods, namely the RoBERTa and IndoBERT architectures. The dataset used is processed in the SQuAD format in the Indonesian language. The collected SQuAD dataset in the Indonesian language consists of 5046 annotated data. The result shows that IndoBERT outperforms RoBERTa with EM and F1-Score values of 81.17 and 91.32, respectively, surpassing RoBERTa with EM and F1-Score values of 79.53 and 90.18.

show abstract

“…Ghiassi et al [ 18 ] present an integrated solution that combines a new clustering algorithm, with a domain transferrable feature engineering approach for Twitter sentiment analysis and spam filtering of YouTube comments. Kim et al [ 19 ] propose a question-answer method to automatically provide users with infrastructure damage information from textual data. Stitini et al [ 20 ] conclude that the linkage between contextual information and classification enhances and improves the recommendation results.…”

Section: Related Workmentioning

confidence: 99%

A Multimodel-Based Deep Learning Framework for Short Text Multiclass Classification with the Imbalanced and Extremely Small Data Set

Tong¹,

Wang²,

Rui³

2022

Computational Intelligence and Neuroscience

View full text Add to dashboard Cite

Text classification plays an important role in many practical applications. In the real world, there are extremely small datasets. Most existing methods adopt pretrained neural network models to handle this kind of dataset. However, these methods are either difficult to deploy on mobile devices because of their large output size or cannot fully extract the deep semantic information between phrases and clauses. This paper proposes a multimodel-based deep learning framework for short-text multiclass classification with an imbalanced and extremely small dataset. Our framework mainly includes five layers: the encoder layer, the word-level LSTM network layer, the sentence-level LSTM network layer, the max-pooling layer, and the SoftMax layer. The encoder layer uses DistilBERT to obtain context-sensitive dynamic word vectors that are difficult to represent in traditional feature engineering methods. Since the transformer part of this layer is distilled, our framework is compressed. Then, we use the next two layers to extract deep semantic information. The output of the encoder layer is sent to a bidirectional LSTM network, and the feature matrix is extracted hierarchically through the LSTM at the word and sentence level to obtain the fine-grained semantic representation. After that, the max-pooling layer converts the feature matrix into a lower-dimensional matrix, preserving only the obvious features. Finally, the feature matrix is taken as the input of a fully connected SoftMax layer, which contains a function that can convert the predicted linear vector into the output value as the probability of the text in each classification. Extensive experiments on two public benchmarks demonstrate the effectiveness of our proposed approach on an extremely small dataset. It retains the state-of-the-art baseline performance in terms of precision, recall, accuracy, and F1 score, and through the model size, training time, and convergence epoch, we can conclude that our method can be deployed faster and lighter on mobile devices.

show abstract

Question answering method for infrastructure damage information retrieval from textual data using bidirectional encoder representations from transformers

Cited by 27 publications

References 27 publications

RETRACTED: Interpreting, Querying, and Securing Complex Financial Documents: A Blockchain-Enabled, OCR & Language Model Approach with Enhanced Cybersecurity Features

RETRACTED: Interpreting, Querying, and Securing Complex Financial Documents: A Blockchain-Enabled, OCR & Language Model Approach with Enhanced Cybersecurity Features

Comparative Analysis of Transformer-Based Method In A Question Answering System for Campus Orientation Guides

A Multimodel-Based Deep Learning Framework for Short Text Multiclass Classification with the Imbalanced and Extremely Small Data Set

Contact Info

Product

Resources

About

Question answering method for infrastructure damage information retrieval from textual data using bidirectional encoder representations from transformers

Cited by 27 publications

References 27 publications

RETRACTED: Interpreting, Querying, and Securing Complex Financial Documents: A Blockchain-Enabled, OCR &amp; Language Model Approach with Enhanced Cybersecurity Features

RETRACTED: Interpreting, Querying, and Securing Complex Financial Documents: A Blockchain-Enabled, OCR &amp; Language Model Approach with Enhanced Cybersecurity Features

Comparative Analysis of Transformer-Based Method In A Question Answering System for Campus Orientation Guides

A Multimodel-Based Deep Learning Framework for Short Text Multiclass Classification with the Imbalanced and Extremely Small Data Set

Contact Info

Product

Resources

About

RETRACTED: Interpreting, Querying, and Securing Complex Financial Documents: A Blockchain-Enabled, OCR & Language Model Approach with Enhanced Cybersecurity Features

RETRACTED: Interpreting, Querying, and Securing Complex Financial Documents: A Blockchain-Enabled, OCR & Language Model Approach with Enhanced Cybersecurity Features