You Don’t Need Labeled Data for Open-Book Question Answering

Gholami, Sia; Noori, Mehdi

doi:10.3390/app12010111

Cited by 13 publications

(4 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our system's precision, recall, and F1-Score are 82.8%, 87%, and 84.8%, respectively, which surpass the precision of 62%, recall of 87%, and F1-Score of 67% reported in other research [32]. The proposed QA system's effectiveness is affirmed by the fact that it surpasses the recall result of other research with 42.70% [33] and outperforms other research [31], [34], [35] in terms of F1-Score, which is 42.6% [31], 49% [34], and 70.8% [35]. This positions it as a leading solution for automatic document processing and information retrieval tasks across a wide range of domains.…”

Section: G Discussionmentioning

confidence: 45%

Retrieval-Augmented Generation Approach: Document Question Answering using Large Language Model

Muludi,

Fitria,

Triloka

et al. 2024

IJACSA

View full text Add to dashboard Cite

This study introduces the Retrieval Augmented Generation (RAG) method to improve Question-Answering (QA) systems by addressing document processing in Natural Language Processing problems. It represents the latest breakthrough in applying RAG to document question and answer applications, overcoming previous QA system obstacles. RAG combines search techniques in vector store and text generation mechanism developed by Large Language Models, offering a time-efficient alternative to manual reading limitations. The research evaluates RAG's that use Generative Pre-trained Transformer 3.5 or GPT-3.5-turbo from the ChatGPT model and its impact on document data processing, comparing it with other applications. This research also provides datasets to test the capabilities of the QA document system. The proposed dataset and Stanford Question Answering Dataset (SQuAD) are used for performance testing. The study contributes theoretically by advancing methodologies and knowledge representation, supporting benchmarking in research communities. Results highlight RAG's superiority: achieving a precision of 0.74 in Recall-Oriented Understudy for Gisting Evaluation (ROUGE) testing, outperforming others at 0.5; obtaining an F1 score of 0.88 in BERTScore, surpassing other QA apps at 0.81; attaining a precision of 0.28 in Bilingual Evaluation Understudy (BLEU) testing, surpassing others with a precision of 0.09; and scoring 0.33 in Jaccard Similarity, outshining others at 0.04. These findings underscore RAG's efficiency and competitiveness, promising a positive impact on various industrial sectors through advanced Artificial Intelligence (AI) technology.

show abstract

Section: G Discussionmentioning

confidence: 45%

Retrieval-Augmented Generation Approach: Document Question Answering using Large Language Model

Muludi,

Fitria,

Triloka

et al. 2024

IJACSA

View full text Add to dashboard Cite

show abstract

“…To examine long-term QA-matching technology based on deep learning for psychological counseling, Chen and Xu [39] improved the matching effect by developing a deep structured semantic model (DSSM) using a bidirectional gate recurrent unit (BiGRU) and a double attention layer. Gholami and Noori [40] presented a new solution for zero-shot open-book QA. Noraset et al [41] developed QA systems using the Bi-LSTM model for Thai users.…”

Section: General Question Answeringmentioning

confidence: 99%

A Question-Answering Model Based on Knowledge Graphs for the General Provisions of Equipment Purchase Orders for Steel Plants Maintenance

Lee

Choi

2023

Electronics

View full text Add to dashboard Cite

Recently, equipment replacement and maintenance repair and operation (MRO) optimization have substantially increased owing to the aging and deterioration of industrial plants, such as steel-making factories in Korea. Therefore, plant owners are required to quickly review equipment supply contracts, i.e., purchase order (PO) documents, with suppliers and vendors. Currently, there is inconsistency in the time and quality required for the PO document review process by engineers, depending on their manual skills and practice. This study developed a general provisions question-answering model (GPQAM) by combining knowledge graph (KG) and question-answering (QA) techniques to search for semantically connected contract clauses through the definition of relationships between entities during the review of equipment purchase contracts. The PO documents analyzed in this case study were based on one steel-making company’s general provisions (GP). GPQAM is a machine learning (ML)-based model with two sub-models (i.e., KG and QA) that automatically generates the most relevant answers to semantic search questions through a cypher query statement in GP for the PO engineers. First, based on the developed GP lexicon and its classifying taxonomy to be stored in the Neo4j graph database (GDB), the KG sub-model finds the corresponding synonyms and consequently shows GP-related information in a graphic form. Second, the QA sub-model is a function to find and answer contract information within the KG and applies pattern-matching technology based on the Aho–Corasick (AC) algorithm. Third, nodes with the meaning most similar to the question are selected using similarity measurement if a response cannot be extracted through the pattern-matching process. Forty-five pilot test questions were created and applied to the GPQAM model evaluation. The F1 score was 82.8%, indicating that the unsupervised training methods developed in this study could be better applied to a semantic QA process in plant engineering documents, where sufficient training data are limited and bargained. An expert survey of PO practitioners confirmed that the semantic QA capability of GPQAM might be efficient and useful for their work. As the first case of applying KG technology to semantic QA for plant equipment PO contracts, this study might be a meaningful contribution to the steel plant industry and, therefore, extended to construction and engineering contract applications.

show abstract

“…Most of these techniques start by extracting the features using machine learning and optimization algorithms, followed by training classifcation models, like a one-class classifer. However, researchers were not developed and customized for the entire issue, and such feature maps are not at their optimal [11][12][13].…”

Section: Introductionmentioning

confidence: 99%

Anomaly Prediction over Human Crowded Scenes via Associate-Based Data Mining and K-Ary Tree Hashing

Yasin

Tahir

Frnda

et al. 2023

International Journal of Intelligent Systems

View full text Add to dashboard Cite

Anomaly detection and behavioral recognition are key research areas widely used to improve human safety. However, in recent times, with the extensive use of surveillance systems and the substantial increase in the volume of recorded scenes, the conventional analysis of categorizing anomalous events has proven to be a difficult task. As a result, machine learning researchers require a smart surveillance system to detect anomalies. This research introduces a robust system for predicting pedestrian anomalies. First, we acquired the crowd data as input from two benchmark datasets (including Avenue and ADOC). Then, different denoising techniques (such as frame conversion, background subtraction, and RGB-to-binary image conversion) for unfiltered data are carried out. Second, texton segmentation is performed to identify human subjects from acquired denoised data. Third, we used Gaussian smoothing and crowd clustering to analyze the multiple subjects from the acquired data for further estimations. The next step is to perform feature extraction to multiple abstract cues from the data. These bag of features include periodic motion, shape autocorrelation, and motion direction flow. Then, the abstracted features are mapped into a single vector in order to apply data optimization and mining techniques. Next, we apply the associate-based mining approach for optimized feature selection. Finally, the resultant vector is served to the k-ary tree hashing classifier to track normal and abnormal activities in pedestrian crowded scenes.

show abstract

You Don’t Need Labeled Data for Open-Book Question Answering

Cited by 13 publications

References 30 publications

Retrieval-Augmented Generation Approach: Document Question Answering using Large Language Model

Retrieval-Augmented Generation Approach: Document Question Answering using Large Language Model

A Question-Answering Model Based on Knowledge Graphs for the General Provisions of Equipment Purchase Orders for Steel Plants Maintenance

Anomaly Prediction over Human Crowded Scenes via Associate-Based Data Mining and K-Ary Tree Hashing

Contact Info

Product

Resources

About