Efficient Deployment of Conversational Natural Language Interfaces over Databases

Colas, Anthony; Bui, Trung; Dernoncourt, Franck; Sinha, Moumita; Kim, Doo Soon

doi:10.18653/v1/2020.nli-1.4

Cited by 6 publications

(9 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…On the other hand, How2QA [37] and iVQA [69] collected questions and answers by presenting videos to crowd workers. In particular, TutorialVQA [13] and PsTuts-VQA [78] focus on software tutorial videos, collecting questions from crowd workers by presenting answer segments or having software experts craft questions. However, since these questions are artificially generated or automatically generated from transcripts, using these questions can be limiting when developing approaches to address questions from real-world users.…”

Section: Video Question Answeringmentioning

confidence: 99%

AQuA: Automated Question-Answering in Software Tutorial Videos with Visual Anchors

Yang,

Vermeulen,

Fitzmaurice

et al. 2024

Proceedings of the CHI Conference on Human Factors in Computing Systems

View full text Add to dashboard Cite

Figure 1: Overall architecture of our question-answer pipeline AQuA, which generates useful responses to questions made in software tutorial videos. Questions are accompanied by visual anchors, which are specific visual elements of interest in the video. The Visual Recognition Module generates a textual description of the visual anchor. Combining the description with the question, the Retrieval Module retrieves relevant articles to the queries. Resources in yellow boxes are software-specific materials (in this case, for Fusion 360). Along with these retrieved articles, the question text, and the visual anchor description, we include the title and relevant transcript sentences of the tutorial video and feed them into GPT-4 through crafted prompts.

show abstract

Section: Video Question Answeringmentioning

confidence: 99%

AQuA: Automated Question-Answering in Software Tutorial Videos with Visual Anchors

Yang,

Vermeulen,

Fitzmaurice

et al. 2024

Proceedings of the CHI Conference on Human Factors in Computing Systems

View full text Add to dashboard Cite

show abstract

“…Audio QA datasets : DAQA [83] on audio temporal reasoning, Clotho-AQA [84] on binary and multichoice audio QA. Video QA datasets : such as VideoQA [85] for multi-domain, MovieQA [86]/MovieFIB [87]/TVQA [88]/KnowIT VQA [89] for movies and shows, MarioQA [90] for games, PororoQA [91] for cartoons, TurorialVQA [92] for tutorials, CLEVRER [93] for physical & causal reasoning. Multi-modal multi-hop QA datasets : MultiModalQA/MMQA [94] for multi-modal and multi-hop QA, WebQA [95] on web multi-modal QA, MAQA focus on negation learning and testing [96].…”

Section: Datasetsmentioning

confidence: 99%

Complex QA and language models hybrid architectures, Survey

Daull¹,

Bellot²,

Bruno³

et al. 2023

Preprint

View full text Add to dashboard Cite

This paper provides a survey of the state of the art of hybrid language models architectures and strategies for "complex" question-answering (QA, CQA, CPS). Very large language models are good at leveraging public data on standard problems but once you want to tackle more specific complex questions or problems you may need specific architecture, knowledge, skills, tasks, methods, sensitive data, performance, human approval and versatile feedback... This survey extends findings from the robust community edited research papers BIG, BLOOM and HELM which open source, benchmark and analyze limits and challenges of large language models in terms of tasks complexity and strict evaluation on accuracy (e.g. fairness, robustness, toxicity, ...). It identifies the key elements used with Large Language Models (LLM) to solve complex questions or problems. Recent projects like ChatGPT and GALACTICA have allowed non-specialists to grasp the great potential as well as the equally strong limitations of language models in complex QA. Hybridizing these models with different components could allow to overcome these different limits and go much further. We discuss some challenges associated with complex QA, including domain adaptation, decomposition and efficient multi-step QA, long form QA, non-factoid QA, safety and multi-sensitivity data protection, multimodal search, hallucinations, QA explainability and truthfulness, time dimension. Therefore we review current solutions and promising strategies, using elements such as hybrid LLM architectures, human-in-the-loop reinforcement learning, prompting adaptation, neuro-symbolic and structured knowledge grounding, program synthesis, and others. We analyze existing solutions and provide an overview of the current research and trends in the area of complex QA.

show abstract

“…Mainly working over Open domain question answering (QA). [1] For this purpose, a huge amount of data corpus is required and from various sources. Traditional approach requires training state of the art models which utilize this data corpus and learn user behavior over voice commands.…”

Section: Related Workmentioning

confidence: 99%

“…The research [1] also shows the use of domain ontology triples which carry this format. <object, relation, property>.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Semantic Parsing to Manipulate Relational Database For a Management System

Mushtaq

2021

Preprint

View full text Add to dashboard Cite

Chatbots and AI assistants have claimed their importance in today's life. The main reason behind adopting this technology is to connect with the user, understand their requirements, and fulfill them. This has been achieved but at the cost of heavy training data and complex learning models. This work is carried out proposes a simple algorithm, a model which can be implemented in different fields each with its own work scope. The proposed model converts human language text to computer-understandable SQL queries. The model requires data only related to the specific field, saving data space. This model performs linear computation hence solving the computational complexity. This work also defines the stages where a new methodology is implemented and what previous method was adopted to fulfill the requirement at that stage. Two datasets available online will be used in this work, the ATIS dataset, and WikiSQL. This work compares the computation time among the 2 datasets and also compares the accuracy of both. This paper works over basic Natural language processing tasks like semantic parsing, NER, parts of speech and tends to achieve results through these simple methods.

show abstract

Efficient Deployment of Conversational Natural Language Interfaces over Databases

Cited by 6 publications

References 20 publications

AQuA: Automated Question-Answering in Software Tutorial Videos with Visual Anchors

AQuA: Automated Question-Answering in Software Tutorial Videos with Visual Anchors

Complex QA and language models hybrid architectures, Survey

Semantic Parsing to Manipulate Relational Database For a Management System

Contact Info

Product

Resources

About