Conversations with Documents: An Exploration of Document-Centered Assistance

Hoeve, Maartje ter; Sim, Robert B.; Nouri, Elnaz; Fourney, Adam; Rijke, Maarten de; White, Ryen W.

doi:10.1145/3343413.3377971

Cited by 23 publications

(8 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Previous studies have found that small screen devices like smartphones are usually used on-the-go, leading to fragmented user attention [4,78]. However, this phenomenon requires further investigation, under various interaction modalities (e.g., smartphone screen vs. voice [97]) and contexts (e.g., walking vs. driving [174]).…”

Section: Conclusion and Discussionmentioning

confidence: 98%

Improving search and recommendation by asking clarifying questions

Zou

2021

SIGIR Forum

View full text Add to dashboard Cite

Recently, there has been renewed interest in interactive search and recommender systems through the development of systems that ask clarifying questions (CQs) to users to better understand their information needs. Although the significance and effectiveness of CQs have been recongnized, research on modelling CQs to improve search and recommender systems is limited, compared to traditional search and recommender systems. Also, understanding users' willingness or ability to answer these CQs, and the extent to which CQs impact users' behavior and their ability to identify relevant information remains relatively unexplored. The work in this thesis provides a series of models and online user studies for search and recommender systems that construct and use CQs. In the first part of the thesis, we focus on CQ-based search and recommendation algorithms. We first propose a novel CQ-based document search algorithm, Sequential Bayesian Search based method for Technology-Assisted Review (SBSTAR) and its extension, to efficiently retrieve the last few, but significant, relevant documents [Zou et al., 2018; Zou and Kanoulas, 2020]. The algorithm sequentially selects and directly asks CQs to users about the presence or absence of an entity in the missing relevant documents. Next, we propose a novel CQ-based product search algorithm, Question-based Sequential Bayesian Product Search (QSBPS), to effectively locate products that users are looking for [Zou and Kanoulas, 2019]. The model is based on a duet learning framework that learns product relevance and the reward of the potential CQ to be asked to the user. Furthermore, we propose a novel CQ-based recommendation algorithm, Question-based recommendation (Qrec), to assist users to find items interactively [Zou et al., 2020a]. The model is first trained offline by a novel matrix factorization algorithm, and then iteratively updates the user and item latent factors online based on the user's answers. We experimentally test the performance of the proposed models and demonstrate that they outperform state-of-the-art baselines. In the second part of the thesis, we focus on the evaluation of CQ-based search and recommendation. We first conduct an online user study by deploying a CQ-based system in the domain of online retail, to understand to what extent users can answer CQs. We explore the user willingness, ability, and user perception to provide answers to CQs [Zou et al., 2020b]. Next, we conduct a large user study on web search to understand the impact of CQ interactions on user search behavior and satisfaction, and how users interact with different levels of quality of CQs under different user perceptions and conditions. Awarded by : University of Amsterdam, Amsterdam, The Netherlands on 6 September 2021. Supervised by : Evangelos Kanoulas, Maarten de Rijke. Available at : https://hdl.handle.net/11245.1/3c46aa78-98af-4997-aef7-44db7ab2fa3d.

show abstract

Section: Conclusion and Discussionmentioning

confidence: 98%

Improving search and recommendation by asking clarifying questions

Zou

2021

SIGIR Forum

View full text Add to dashboard Cite

show abstract

“…The questions would be different from factoid questions and chitchat, because here the focus would be on relevant information from that specific document. 65 This category seems to have a big scope for understanding, reviewing, and inferring knowledge from documents.…”

Section: Existing Accessibility-driven Solutions For Pdf Documentsmentioning

confidence: 99%

Accessibility of Tables in PDF Documents

Fayyaz

Khusro

Ullah

2021

ITAL

View full text Add to dashboard Cite

People access and share information over the web and in other digital environments, including digital libraries, in the form of documents such as books, articles, technical reports, etc. These documents are in a variety of formats, of which the Portable Document Format (PDF) is most widely used because of its emphasis on preserving the layout of the original material. The retrieval of relevant material from these derivative documents is challenging for information retrieval (IR) because the rich semantic structure of these documents is lost. The retrieval of important units such as images, figures, algorithms, mathematical formulas, and tables becomes a challenge. Among these elements, tables are particularly important because they can add value to the resource description, discovery, and accessibility of documents not only on the web but also in libraries if they are made retrievable and presentable to readers. Sighted users comprehend tables for sensemaking using visual cues, but blind and visually impaired users must rely on assistive technologies, including text-to-speech and screen readers, to comprehend tables. However, these technologies do not pay sufficient attention to tables in order to effectively present tables to visually impaired individuals. Therefore, ways must be found to make tables in PDF documents not only retrievable but also comprehensible. Before developing such solutions, it is necessary to review the available assistive technologies, tools, and frameworks for their capabilities, strengths, and limitations from the comprehension perspective of blind and visually impaired people, along with suitable environments like digital libraries. We found no such review article that critically and analytically presents and evaluates these technologies. To fill this gap in the literature, this review paper reports on the current state of the accessibility of PDF documents, digital libraries, assistive technologies, tools, and frameworks that make PDF tables comprehensible and accessible to blind and visually impaired people. The study findings have implications for libraries, information sciences, and information retrieval.

show abstract

“…BVIP should be able to satisfy their information needs without having to sequentially go through all the website content and structure -a process that can be time consuming and frustrating for screen reader users [19]. This support is rooted in the ongoing efforts in conversational Q&A [13] and document-centered digital assistant [18]. The idea is to support BVIP users to perform natural language queries (NLQ) on the contents of websites, and to inquire about the properties defined in the website's metadata.…”

Section: Characterising Conversational Browsing Supportmentioning

confidence: 99%

Bringing Cognitive Augmentation to Web Browsing Accessibility

Pina¹,

Báez²,

Daniel³

2020

Preprint

View full text Add to dashboard Cite

In this paper we explore the opportunities brought by cognitive augmentation to provide a more natural and accessible web browsing experience. We explore these opportunities through conversational web browsing, an emerging interaction paradigm for the Web that enables blind and visually impaired users (BVIP), as well as regular users, to access the contents and features of websites through conversational agents. Informed by the literature, our previous work and prototyping exercises, we derive a conceptual framework for supporting BVIP conversational web browsing needs, to then focus on the challenges of automatically providing this support, describing our early work and prototype that leverage heuristics that consider structural and content features only.

show abstract

Conversations with Documents: An Exploration of Document-Centered Assistance

Cited by 23 publications

References 35 publications

Improving search and recommendation by asking clarifying questions

Improving search and recommendation by asking clarifying questions

Accessibility of Tables in PDF Documents

Bringing Cognitive Augmentation to Web Browsing Accessibility

Contact Info

Product

Resources

About