Evaluating the Use of Digital Creativity Support by Journalists in Newsrooms

Abstract

This paper reports the evaluation of a new digital support tool designed to increase journalist creativity and productivity in newsrooms. After outlining the tool's principles, interactive features and architecture, the paper reports the installation and use of the tool over 2 months by 12 journalists in the newsrooms of 3 newspapers. Results from this evaluation revealed that tool use was associated with published news articles rated as more novel but not more valuable than published articles written by the same journalists without the tool. However, tool use did not increase journalist productivity. The evaluation results were used to inform future changes to the digital creativity support tool.

Creative Information Exploration in Journalism

“…The use of a 3-tier architecture has enabled cost-effective adaptation of the tool for use in different settings: the distinct interface layer has enabled new versions of INJECT to be delivered quickly for different text editor and web application environments; the separate application layer has been enabled these new versions of INJECT to have most if not all of the reasoning and interaction capabilities of INJECT at launch; and the separate data layer has enabled configurations of the INJECT tool to manipulate news and social media information from different private and public sources and written in different languages. Early evaluation of INJECT [14] by a limited number of journalists revealed that the tool was effective in supporting journalists discover new and useful angles on stories written with INJECT's support, often quickly. However, the tool has been more effective when extended existing storylines rather than developed new ones.…”

AI should embody our values: Investigating journalistic values to inform AI technology design

“…INJECT [43,44] supported story creation through creative idea generation by using machine learning and NLP to extract terms from a partially-written story. It then used the extracted terms to search news sources for existing stories to inspire the in-progress story.…”

“…The AI cards were designed in response. The AI cards were inspired by existing algorithmic journalism tools [11,20,43,44,65] (see section 2.3).…”

“…We decided not to describe existing systems (e.g. INJECT [44]) as we wanted to focus the discussion on the values potentially impacted by a range of different types of AI systems rather than on the value-related impact of specific systems or functionality. Thus, the cards acted as open-ended rough sketches of realistic types of AI system.…”

“…In the US, Associated Press has produced more than 3,700 stories using algorithms [20]. Scholars have also created AI systems for newsrooms, supporting investigative research through data analysis and visualisation [9,11,65], automatic story generation based on newsworthiness [41] and story-related creativity [43,44] . Newsrooms are cautiously optimistic about the potential of AI and plan to step up its use to drive more effective content distribution and achieve time and cost efficiencies [5,48].…”