An Interface for Agent Supported Conversational Search

Kaushik, Abhishek; Ramachandra, Vishal Bhat; Jones, Gareth J. F.

doi:10.1145/3343413.3377942

Cited by 12 publications

(26 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It should be noted, however, that our findings need to be considered in context. We evaluated one particular type of CSA -a chat/text based CSA like those proposed in [6,32,67] -where we employed the traditional IR evaluation approach in a conversational setting. We also used simulation based methodology so that we could begin to explore the large evaluation space (which would be near impossible to do so within a user study).…”

Section: Discussion and Future Workmentioning

confidence: 99%

“…), provides many results, etc., while (iii) a chat based CSA agent has some restrictions on screen space and bandwidth within the chat window, but has the advantage over voice only CSAs because the conversation is persistent so the user can refer back to options, etc.. For the purposes of this work, we will assume that the CSA is a chat-based CSA like that in Fig. 1 which represents the interfaces explored in [6,32,67]. Of note is that depending on the interface and its modality, the cost of different conversational turns will vary -and this will impact on how much gain the user accumulates from their conversation with the agent.…”

Section: Tell About His Career Before Being Presidentmentioning

confidence: 99%

“…Zhang et al [72] proposed a System Ask, User Respond paradigm, which is akin to query-refinement, where after the initial request is made, the agent will ask for refinements/clarifications until it is confident enough to present results. Kaushik et al [32] presented a system sided workflow model consisting of the steps the agent takes when dealing with a user's request (e.g. handling greetings and error handling).…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Analysing Mixed Initiatives and Search Strategies during Conversational Search

Aliannejadi

Azzopardi

Zamani

et al. 2021

Proceedings of the 30th ACM International Conference on Information &Amp; Knowledge Management

View full text Add to dashboard Cite

Information seeking conversations between users and Conversational Search Agents (CSAs) consist of multiple turns of interaction. While users initiate a search session, ideally a CSA should sometimes take the lead in the conversation by obtaining feedback from the user by offering query suggestions or asking for query clarifications i.e. mixed initiative. This creates the potential for more engaging conversational searches, but substantially increases the complexity of modelling and evaluating such scenarios due to the large interaction space coupled with the trade-offs between the costs and benefits of the different interactions. In this paper, we present a model for conversational search -from which we instantiate different observed conversational search strategies, where the agent elicits: (i) Feedback-First, or (ii) Feedback-After. Using 49 TREC WebTrack Topics, we performed an analysis comparing how well these different strategies combine with different mixed initiative approaches: (i) Query Suggestions vs. (ii) Query Clarifications. Our analysis reveals that there is no superior or dominant combination, instead it shows that query clarifications are better when asked first, while query suggestions are better when asked after presenting results. We also show that the best strategy and approach depends on the trade-offs between the relative costs between querying and giving feedback, the performance of the initial query, the number of assessments per query, and the total amount of gain required. While this work highlights the complexities and challenges involved in analyzing CSAs, it provides the foundations for evaluating conversational strategies and conversational search agents in batch/offline settings. CCS CONCEPTS• Information systems → Users and interactive retrieval; • Human-centered computing → HCI design and evaluation methods.

show abstract

Section: Discussion and Future Workmentioning

confidence: 99%

Section: Tell About His Career Before Being Presidentmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Analysing Mixed Initiatives and Search Strategies during Conversational Search

Aliannejadi

Azzopardi

Zamani

et al. 2021

Proceedings of the 30th ACM International Conference on Information &Amp; Knowledge Management

View full text Add to dashboard Cite

show abstract

“…The study by Aliannejadi et al [113] highlights different conversational search strategies used by an agent that guide the search by providing query suggestions and clarifications. Kaushik et al [19] introduce a conversational agent to assist users in performing search tasks. The multi-view search interface includes a graphical interface and a conversational agent to help the user to perform searches without increasing their cognitive load.…”

Section: Deep Learning and Machine Learning Based Systemsmentioning

confidence: 99%

“…The focus rests on providing a natural conversational pattern so the system appears human-like. Kaushik et al [19] introduce a conversational search system with a dialog agent that assists users in complex information seeking processes. QA systems can be used for tasks such as answering questions about university admissions [20], quiz generation [21], customer service [22], and more.…”

Section: Introductionmentioning

confidence: 99%

Do You Ever Get Off Track in a Conversation? The Conversational System’s Anatomy and Evaluation Metrics

Yadav

Kaushik

2022

Knowledge

Self Cite

View full text Add to dashboard Cite

Conversational systems are now applicable to almost every business domain. Evaluation is an important step in the creation of dialog systems so that they may be readily tested and prototyped. There is no universally agreed upon metric for evaluating all dialog systems. Human evaluation, which is not computerized, is now the most effective and complete evaluation approach. Data gathering and analysis are evaluation activities that need human intervention. In this work, we address the many types of dialog systems and the assessment methods that may be used with them. The benefits and drawbacks of each sort of evaluation approach are also explored, which could better help us understand the expectations associated with developing an automated evaluation system. The objective of this study is to investigate conversational agents, their design approaches and evaluation metrics. This approach can help us to better understand the overall process of dialog system development, and future possibilities to enhance user experience. Because human assessment is costly and time consuming, we emphasize the need of having a generally recognized and automated evaluation model for conversational systems, which may significantly minimize the amount of time required for analysis.

show abstract