A Methodological Proposal to Evaluate Journalism Texts Created for Depopulated Areas Using AI

Abstract

The public service media Radio Televisión Española (RTVE) conducted a proof-of-concept study to automatically generate reports on the results of the local elections of 28 May 2023 in Spanish communities with fewer than 1000 inhabitants. This study describes the creation, testing and application of the methodological tool used to evaluate the quality of the reports generated using artificial intelligence in order to optimize the algorithm. The application of the proposed datasheet provided a systematic analysis, and the iterative use of the tool made it possible to gradually improve the results produced by the system until a suitable threshold was reached for publication. The study also showed that, despite the ability of AI systems to automatically generate a large volume of information, both human labour and the reliability of the data that feed the system are essential to ensure journalistic quality.