Data science training for official statistics: A new scientific paradigm of information and knowledge development in national statistical systems

Abstract

The ability to incorporate new and Big Data sources and to benefit from emerging technologies such as Web Technologies, Remote Data Collection methods, User Experience Platforms, and Trusted Smart Statistics will become increasingly important in producing and disseminating official statistics. The skills and competencies required to automate, analyse, and optimize such complex systems are often not part of the traditional skill set of most National Statistical Offices. The adoption of these technologies requires new knowledge, methodologies and the upgrading of the quality assurance framework, technology, security, privacy, and legal matters. However, there are methodological challenges and discussions among scholars about the diverse methodical confinement and the wide array of skills and competencies considered relevant for those working with big data at NSOs. This paper develops a Data Science Model for Official Statistics (DSMOS), graphically summarizing the role of data science in statistical business processes. The model combines data science, existing scientific paradigms, and trusted smart statistics, and develops around a restricted number of constructs. We considered a combination of statistical engineering, data engineering, data analysis, software engineering and soft skills such as statistical thinking, statistical literacy and specific knowledge of official statistics and dissemination of official statistics products as key requirements of data science in official statistics. We then analyse and discuss the educational requirements of the proposed model, clarifying their contribution, interactions, and current and future importance in official statistics. The DSMOS was validated through a quantitative method, using a survey addressed to experts working at the European statistical systems. The empirical results show that the core competencies considered relevant for the DSMOS include acquisition and processing capabilities related to Statistics, high-frequency data, spatial data, Big Data, and microdata/nano-data, in addition to problem-solving skills, Spatio-temporal modelling, machine learning, programming with R and SAS software, Data visualisation using novel technologies, Data and statistical literacy, Ethics in Official Statistics, New data methodologies, New data quality tools, standards and frameworks for official statistics. Some disadvantages and vulnerabilities are also addressed in the paper.

An ensemble learning strategy for panel time series forecasting of excess mortality during the COVID-19 pandemic

“…Now, the correlation coefficient increased dramatically to 94% ( -value 0.000), which can be attributed to the higher quality of the official statistics in these countries. Ashofteh and Bravo (2020) have shown there to be significant variation in the quality of the COVID-19 datasets reported worldwide, albeit a recent study suggests that data science and new technologies can be expected to play a significant role in improving data quality from national statistical offices in the future [52] .…”

“…Ashofteh and Bravo (2020) showed that there is significant variation in the quality of COVID-19 datasets reported worldwide. A recent study suggests that data science and new technologies are expected to play a significant role in improving data quality at National Statistical Offices in the future [53].…”

Research on hospital information data statistics under the background of big data and artificial intelligence

“…With the development of information technology, big data has become an indispensable tool for processing and analyzing large amounts of data [8][9]. It becomes particularly important to find patterns or screen out valuable information in redundant and mixed massive information [10][11].…”

Answering Current Challenges of and Changes in Producing Official Time Use Statistics Using the Data Collection Platform MOTUS

“…Today, National Statistical Institutes (NSIs) face challenges and changes in the way they produce official statistics (Radermacher 2020). On the one hand, technological developments create the opportunity for paradigm shifts in methodology (Ashofteh and Bravo 2021). On the other hand, modern societal changes and challenges create new user demands for high-quality data and statistics (Cai and Zhu 2015).…”