Hitiqa

Small, Sharon; Strzalkowski, Tomek; Liu, Ting; Ryan, Sean; Salkin, Robert; Sato, Nobuyuki; Kantor, Paul B.; Kelly, Diane; Rittman, Robert; Wacholder, Nina

doi:10.3115/1220355.1220544

Cited by 15 publications

(3 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Intelligence is more than data aggregation and hypothesis formation. Automatic tools assist intelligence analysts in data collection [5][6][7], data inspection and visualization [8], evidence credibility assessment [9,10], scenariobased question answering [11], hypotheses generation [12,13], and quality evaluation [14]. However, most current tools still require expensive crowdsourcing [15,16] or human-machine collaboration [17][18][19] in report generation.…”

Section: Related Workmentioning

confidence: 99%

SmartBook: AI-Assisted Situation Report Generation

Reddy¹,

Fung²,

Zeng³

et al. 2023

Preprint

View full text Add to dashboard Cite

Emerging events, such as the COVID pandemic and the Ukraine Crisis, require a time-sensitive comprehensive understanding of the situation to allow for appropriate decision-making and effective action response. Automated generation of situation reports can significantly reduce the time, effort, and cost for domain experts when preparing their official human-curated reports. However, AI research toward this goal has been very limited, and no successful trials have yet been conducted to automate such report generation. Pre-existing natural language processing methods, large language model based text generation, and information retrieval techniques are insufficient to identify, locate, and summarize important information, and lack detailed, structured, and strategic awareness. We propose SmartBook, a novel task formulation targeting situation report generation, which consumes large volumes of news data to produce a structured situation report with multiple hypotheses (claims) summarized and grounded with rich links to factual evidence. We realize SmartBook for the Ukraine-Russia crisis by automatically generating intelligence analysis reports to assist expert analysts. The machine-generated reports are structured in the form of timelines, with each timeline organized by major events (or chapters), corresponding strategic questions (or sections) and their grounded summaries (or section content). Our proposed framework automatically detects realtime event-related strategic questions, which are more directed than manually-crafted analyst questions, which tend to be too complex, hard

show abstract

Section: Related Workmentioning

confidence: 99%

SmartBook: AI-Assisted Situation Report Generation

Reddy¹,

Fung²,

Zeng³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…(2005) and Small et al . (2004) (see Strzalkowski and Harabagiu, 2006 for an overview of advances in analytic QA and descriptions of other research efforts).…”

Section: Introductionmentioning

confidence: 99%

“…Wacholder et al . (2007) describe a series of 3-day evaluations of HITIQA (Small et al ., 2004), a complex QA system designed to support intelligence analysts' exploratory research for analytical questions. Analysts used HITIQA to prepare reports on various scenarios and participated in a number of evaluation exercises including questionnaires, interviews, and report ratings.…”

Section: Introductionmentioning

confidence: 99%

Questionnaires for eliciting evaluation data from users of interactive question answering systems

et al. 2009

View full text Add to dashboard Cite

Evaluating interactive question answering (QA) systems with real users can be challenging because traditional evaluation measures based on the relevance of items returned are difficult to employ since relevance judgments can be unstable in multi-user evaluations. The work reported in this paper evaluates, in distinguishing among a set of interactive QA systems, the effectiveness of three questionnaires: a Cognitive Workload Questionnaire (NASA TLX), and Task and System Questionnaires customized to a specific interactive QA application. These Questionnaires were evaluated with four systems, seven analysts, and eight scenarios during a 2-week workshop. Overall, results demonstrate that all three Questionnaires are effective at distinguishing among systems, with the Task Questionnaire being the most sensitive. Results also provide initial support for the validity and reliability of the Questionnaires.

show abstract

Untitled

2007

J. Am. Soc. Inf. Sci.

486

View full text Add to dashboard Cite

In this article we discuss re-retrieving personal information objects and relate the task to recovering from lapse(s) in memory. We propose that fundamentally it is lapses in memory that impede users from successfully re-finding the information they need. Our hypothesis is that by learning more about memory lapses in non-computing contexts and how people cope and recover from these lapses, we can better inform the design of PIM tools and improve the user's ability to re-access and re-use objects. We describe a diary study that investigates the everyday memory problems of 25 people from a wide range of backgrounds. Based on the findings, we present a series of principles that we hypothesize will improve the design of personal information management tools. This hypothesis is validated by an evaluation of a tool for managing personal photographs, which was designed with respect to our findings. The evaluation suggests that users' performance when re-finding objects can be improved by building personal information management tools to support characteristics of human memory.

show abstract

Hitiqa

Cited by 15 publications

References 8 publications

SmartBook: AI-Assisted Situation Report Generation

SmartBook: AI-Assisted Situation Report Generation

Questionnaires for eliciting evaluation data from users of interactive question answering systems

Untitled

Contact Info

Product

Resources

About